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Preface 


High trade costs hamper the economie potential of many of the poorest nations, pricing 
them out of global markets. Trade costs also fall disproportionately heavily on small- and 
medium-sized enterprises. These companies, which are considered engines of employment 
offering a route out of poverty, are hit hardest by non-tariff measures. A study by the 
International Trade Centre on how businesses experience non-tariff measures in 
23 developing countries has recently confirmed this once more. 

An important step towards reducing trade costs was taken in 2013 at the Ninth WTO 
Ministerial Conference in Bali where members concluded the Agreement on Trade 
Facilitation. Implementation of the Agreement promises to bring substantial cost 
reductions for traders in developing and least developed countries (LDCs), thereby 
supporting increased economie activity, and potentially increasing government revenues. 
The Dutch government has already been actively committed to achieving Aid-for-Trade 
objectives since the early 1970S, by assisting developing countries in reducing non-tariff 
barriers, particularly through the efforts of the Centre for the Promotion of Imports from 
Developing Countries (CBI). 

This report, for the First time, presents the main results of an evaluation on the effect and 
impact of CBI’s Aid-for-Trade interventions on the business sector in developing countries. 
Findings were based on surveys of assisted companies in developing countries, four 
in-depth country case studies, a benchmarking exercise, a literature review and stakeholder 
interviews. The evaluation covers the period 2005-2012 and focuses on the backbone of the 
CBI programme, which consists of technical assistance (export coaching) to individual 
companies in order to strengthen their competitive edge in European markets, and of 
improvement of business support organisations in developing countries. 

The report concludes that CBI has been successful in helping companies to overcome a lack 
of market information and market entiy barriers. This was however not always sufficiënt to 
increase exports. In some cases, observed export increases could not always be (fully) 
attributed to CBI programmes and activities. CBI’s contribution is therefore more modest 
when set against the ultimate intended trade performance goals. 

The evaluation has been carried out under the responsibility of IOB evaluator Peter Henk 
Eshuis, Rob Kuijpers (research assistant) and Albert Stuijfzand (controller). The following 
persons participated in the consultants' team for the evaluation: Phil Compernolle, Sharon 
van Ede, Philip de long (APE, country case studies, surveys and benchmark). The following 
other persons have contributed to the underlying case studies: Peter Hurst (country study 
South-Africa), Oscar Huertas Diaz (country study Colombia), Magda Ghonem, Emad El 
Shaarawy, Sally El Hawary (country study Egypt), Shubhashis Gangopadhyay, Monica Dutta, 
Bhavya Paliwal and Manisha G. Singh (country study India). 
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A reference group chaired by the former director of IOB, Ruerd Ruben, provided valuable 
comments and advice. The group consisted of Clemens Lutz (University of Groningen), 
Remco Oostendorp (VU University), Marcel Vernooij, Henny de Vries, Hajo Provó Kluit 
(Department of Sustainable Economie Development, MFA), Micha van Lin (FME), and Dick 
de Man (CBI). Antonie de Kemp, Piet de Lange and Otto Genee (IOB) acted as peer reviewers 
oftheevaluation. 

IOB would sincerely like to thank CBI, all involved companies, policy officers at the Ministry 
and other stakeholders for the information they provided and their constructive feedback to 
draft versions of the report. 


Dr. Wendy Asbeek Brusse 

Director Policy and Operations Evaluation Department (IOB) 
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Private sector development (PSD) has become a central theme in international cooperation 
policy. By strengthening the entrepreneurial climate in developing countries, donors and 
recipiënt countries aim to create better opportunities for private investment, resulting in 
more employment opportunities and less poverty. In the Netherlands, the government is 
actively committed to achieving these objectives. An important instrument in Dutch PSD 
policies is the Centre for the Promotion of Imports from Developing Countries (CBI). CBI is 
one of the organisations that receive a flnancial contribution from the PSD market 
development cluster of the Ministry of Foreign Affairs (MFA). The cluster promotes inclusive 
economie growth through greater investment, a less informal economy, decent jobs, 
increased productivity and more competitive enterprises. 

When CBI started its activities in the early 1970S, the world was far less connected than 
today. A smaller share of the population, for example, was able to reap the benefits of 
information technology, which lowered the costs of trade and transport. While there is still 
a need to reduce information asymmetry for exporters from developing countries in the 
field of private sector development, and more specifically in trade promotion, an increasing 
number of export promoting organisations (EPOs) offer services that are quite similar across 
countries. Many companies, assisted by CBI, also use other support service organisations. 
The dynamic global context continuously creates challenges that have required CBI to adapt 
its strategy. From 2011 onwards, CBI reformulated its interventions into a more integrated 
approach. The current impact evaluation has not looked into CBI’s recent mode of 
operation, simply because few of the interventions started since then have concluded, 
which means that any evaluation of the effects of these interventions would be 
inconclusive. 

This report presents the results of an evaluation of the effectiveness of CBI. The evaluation 
covers the period 2005-2012 and focuses on the backbone of the CBI programme: (1) the 
sectoral export coaching programme (ECP), which renders technical assistance to individual 
companies in order to strengthen their competitive edge in European markets; and (2) the 
business support organisation development (BSOD), which aims to improve the business 
support organisations' (BSOs) export development and promotion activities in Europe. 

The evaluation of the ECP programme is based on insights gained from using qualitative 
evaluation methods (literature review, document analysis, interviews, and case studies in 
Colombia, Egypt, India, and South Africa), combined with quantitative methods (surveys 
among exporters and importers, statistical analyses of programme monitoring data, and a 
benchmark of CBI’s efficiency). The evaluation of the BSOD programme is predominantly 
based on a review of regular programme evaluations conducted on behalf of CBI. 
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Findings 

Based on the available materials and evidence, the Policy and Operations Evaluation 
Department (IOB) has established the following findings: 

The main evaluation question is: 'To what extern is the CBI programme helping to achieve 
Dutch private sector development (PSD) objectives?’. The CBI programme is consistent with 
the Ministry of Foreign Affairs’ general PSD policy objective. Also, its intervention logic can 
be seen as a refmement of the general policies established in the area of market 
development. CBI’s contribution is less convincing, however, when set against the ultimate 
goals of the PSD policy, i.e. inclusive economie growth through greater investment, a less 
informal economy, decent jobs, increased productivity and more competitive enterprises. 
This is partly attributable to the way CBI has implemented its programmes, but it is also 
reflected in the discrepancy between CBI’s mission statement and MFA's PSD policy 
objectives. Moreover, interventions take time to mature and need to have attained a certain 
scale to have an impact on these PSD aims, especially at the meso-level and macro-level. 

More detailed findings are presented hereafter: 

i) CBI has been successful in supporting exporting companies to overcome their lack ofinformation on 

import markets. However, for many companies this was not sufficiënt to start exporting to Europe. 

CBI has been successful in helping companies to overcome a lack of market information 
and intelligence. The ECP programmes trained and supported 86% of its subscribers; 14% 
dropped out. An analysis of 48 companies shows that almost half of them (21) managed to 
increase their exports. However, taking into account CBI’s own targets, the success rate was 
lower. CBI aimed to achieve an additional export value of 2.5 times the amount it invested 
since the start of the intervention at the programme level. Given the average costs of EUR 
58,000, this is consistent with an additional export turnover of at least EUR 145,000. The 
analysis shows that 16 out of 88 companies (for which data were available) managed to 
achieve this target within two years after completing the programme. 1 Given statistical error 
margins and a dropout rate of 14%, this means that 10-20% of the initial participants 
achieved the CBI target. There is little information, within CBI or in the literature, on the 
sustainability of the effects brought about by promoting export. 

A potential reason why the success rate is not higher, even though CBI was successful in 
bringing companies into contact with buyers, is that a lack of market information is only 
one of the major export constraints. CBI’s interventions only focused on a limited number 
of export constraints, and in general did not address other constraints that are preventing 
companies from exporting. Moreover, the invested resources were not always optimally 
directed at companies according to their needs and existing capacities. Though CBI was 
sensitive to the idea that the enabling environment for private sector development matters, 
by taking a sector ECP as the basic unit for programming its services, it did not necessarily 

1 CBI measures the export turnover from the start of the CBI intervention until one year after completion 
of the programme. 
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take adequate account of the importance of the country context of participating companies. 
Target companies did not always feel that CBI’s services completely met their specifk needs, and 
also feit that CBI could have expressed more interest in the process. Even though CBI followed a 
more proactive course leading to more structured cooperation with international and national 
PSD organisations, some case study countries still viewed CBI as a ‘standalone’ service provider. 

2 ) The criteria for selecting target companies and target coiintries are not clear. Procedures for identifying 
and selecting participants were often unavailable. And when available, these were not always followed. 
Larger, well-developed companies often received the most intensive and expensive treatment (‘picking 
the winners or low-hanging fruit'). Companies obviously more in need received less support. 

CBI selects both stronger and weaker companies for ECP programmes, assuming that the 
stronger companies will serve as an example for the weaker ones. In addition, companies 
from countries with a weaker export profile or image were expected to benefit from 
improved visibility and the reputation of stronger countries, for example in a collective 
trade fair. Decisions on company selection, tailoring the types of intervention to 
participants and assessments about export competence are not subject to checks and 
balances in CBI’s system. 

CBI spent 3% of the total expenditure in low-income countries, while 7% was spent in 
upper-middle-income countries. The companies that benefited most from CBI’s support are 
predominantly from lower-middle-income countries. Interventions in low-income 
countries were (necessarily) overly focused on 'business development’ in general, and left 
insufficiënt room for introducing ‘market access’ modules that should complement the 
range of sub-interventions that prepare companies for international market access. 

3) BSOD evaluations reveal that in general the interventions were relevant, but that there is hardly any 
information on their effectiveness. 

One challenge facing private sector development institutions is how to achieve operational 
scale and gain access to small and medium enterprises (SMEs) in developing countries, 
which by nature are small, diffuse, numerous and difficult to reach. CBI’s Business Support 
Organisation Development programme (BSOD) aims to increase the reach ofits trade- 
related support among SMEs in developing countries, by focusing on strengthening 
intermediary trade support organisations. Generally, the BSOD interventions reviewed were 
considered to be relevant. However, with the increase in the number of active trade 
promotion organisations in developing countries, the relevance and additionality of CBI’s 
interventions could be at stake. Given the heterogeneity of local partners in the BSOD 
programme, it strongly depends on the mandate of the beneficiary, which underlines the 
need for a context-specific and flexible approach in order to meet the needs of BSOs. 

The effects of technical assistance for BSOs are hard to measure accurately. One reason is 
that the quality and result indicators set by MFA and CBI are too general to measure 
increased capacity outcome or even impact. Moreover, the lack of data on the effects of 
BSOD programmes also makes it difficult to monitor and evaluate these programmes and 
steer them in the desired direction. Good coordination between service providers is often 
lacking, leading to the absence or the suboptimal division of tasks between donors. 
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4) Avaüable evidence does not point to an efficiently operating CBI. The right incentives were not in place. 

During the evaluation period, the overhead costs needed to run CBI's programmes 
increased more quickly than the programme budget itself. The average costs per company 
amounted to EUR 58,000 and increased substantially after the introduction of an ex ante 
transfer pricing settlement with MFA. More specifically, since becoming an independent 
agency in 1998, CBI has struggled to set and justify cost prices for its products. A results- 
oriented management model that was supposed to increase the organisations’ efficiency 
was not implemented. The introduction of an ex ante cost price model and the modular 
programme delivery, in 2006 and 2010 respectively, caused CBI’s turnover and profit rate to 
go up, with the profits increasing even more rapidly than the turnover. As a result, CBI 
started to pay back part of its accumulated excess Capital to MFA, and also introduced 
discounts on its tariffs agreed with MFA. 

A comparison between CBI and foreign counterpart organisations suggests that CBI has the 
highest cost level per exporten This may be caused by scale disadvantages and 
fragmentation. CBI has a lower number of participants per programme, implying higher 
fixed costs per participant, while the geographical spread of the participants is much larger, 
implying higher transaction costs per beneficiary. In addition, the inclusion of a 
heterogeneous group of companies, with rather different characteristics as expressed by 
diverging audit scores in one and the same programme, may be inefficiënt. 

5) The Minisoy of Foreign Affairs was not actively involved in CBI’s strategie chokes and operations. As a 
result, the organisation operated relatively independently from the ministry. Since 2010, however, the 
Minisoy of Foreign Affairs has become more prescriptive in what it expects front the organisation. 

While the CBI programme is consistent with the Ministry of Foreign Affairs' general PSD 
policy objectives, the ministry was insufficiently engaged with CBI on priorities and results, 
and did not provide the right incentives. Even though MFA did not have detailed 
information about CBI's cost prices, it approved requests for budget increases, which were 
not always well-argued or evidence-based. 

CBI has operated within its mandate and in accordance with the list of countries approved 
by MFA. The focus was not on the poorest countries, but on countries that were sufficiently 
developed to contribute businesses on the verge of exporting to Europe. 

In 2010, MFA suggested to CBI that it should strengthen the connection between export 
promotion, sustainable development and poverty reduction, and make this more 
transparent in its operations. CBI subsequently developed a new strategy that would focus 
more on least-developed and fragile States and enhance its impact and added value. MFA 
granted CBI's request for a budget increase, which CBI said it needed to compensate for the 
losses and intensified interventions in low-income countries. Expenditures have become 
more aligned to MFA’s demands since that time. However, during the period under review, 
CBI did not become more active in low-income countries and fragile States. Therefore, it 
appears that the budget increase was premature. The budget increase was one of the 
reasons for CBI’s rapid accumulation of Capital. 
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6 ) CBI’s monitoring & evaluation system needs to befunher improved, becctuse, as itstands, it is difficult to 
monitor, steer and evaluate the programme effectively, and settle the budget. 

For the period under consideration, the programme management system did not meet the 
Standard for result measurement and evaluation, in spite of improvements over the years. 
Recommendations from earlierevaluations (e.g. the introduction of a more meaningful set 
of performance indicators) were only partially adopted. No data was being systematically 
collected to establish a baseline and end-line situation for CBI’s interventions. Evaluation 
outcomes were predominantly based on participants’ self-perception. The evaluations 
conducted on behalf of CBI did not systematically address issues such as contribution, 
additionality and sustainability. 

The assessment of the ECP and BSOD programmes shows that CBI lacked detailed insight 
into its own programme results, as well as on the underlying explanatory reasons for these 
results. The programme management data did not provide information on export figures 
for 20% of the companies declared by CBI as ‘competent exporters’ (which was the base for 
Financial settlement with MFA), as it should according to the agreed quality indicators for 
result, effect and impact. 


Lessons for the future 

Doing the right things 

In terms of whether CBI has been doing the right things, there are a number of areas that 
can be improved at the level of programme design and implementation. 

Improved needs assessment should increase the number of tailor-made interventions 

When looking at the profiles of assisted companies, it becomes clear that CBI has been 
servicing a heterogeneous group of companies, in terms of country of origin, sector and 
different stages of export development and internationalisation. This implies that 
companies need different types of services and interventions, whereas the programme in its 
current set-up offers Standard service modules for all participants, without addressing other 
limiting constraints that hold these companies back from exporting or exporting more. 

Narrow the focus and reduce the number ofeligible countries 

One challenge confronting private sector development institutions is how to achieve 
operational scale when providing support to SMEs in developing countries, which by nature 
are small, diffuse and numerous. The programme could benefit from a more well-reasoned 
selection policy of eligible countries that restricts selection to low- and lower-middle- 
income countries. (This should also reduce the number of eligible countries.) Selectivity 
will also most likely result in lower transaction costs per beneficiary and hence more 
efficiency. With the introduction of the integrated approach starting in 2011, CBI has already 
entered the path of enhanced selectivity. The selection should also look closely at the 
country’s context, making use of existing sources of public information. 
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The portfolio reveals that the risk of ending up with an underqualified company is high in 
low-income countries. These countries may benefit more from business development than 
from Standard export coaching programmes. Therefore, in order to enhance effectiveness in 
these countries, CBI could work more closely together with business development 
organisations and farmer support organisations. 

Enhancing sustainability by dejïning an exitstrategy 

Starting an intervention does not only require clear commitments between the contract 
parties, but also requires good management of expectations and clear thinking about how 
to sustain the intervention results. During the evaluation period, CBI had no formal Systems 
in place to enhance the sustainability of its support to individual companies. As other 
evaluators have remarked in the past, BSOD interventions in particular could profit from an 
‘exit strategy’. This would create trust and confidence with the counterparts and their 
stakeholders, who have to develop a certain confidence in the future capacity of the BSO 
without CBI’s support. 

Enhance monitoring, evaluation, and steering ofprogramme interventions 

CBI is currently developing a real-time impact monitoring and evaluation system that 
should address the current lack of effective programme monitoring and evaluation data, 
which makes it difficult to steer the programme effectively. As this is a work in progress, our 
advice is to integrate quantitative, research-based impact assessments - such as those based 
on randomised controlled trials or quasi-experimental approaches - into programmes from 
the design phase. In order to be applicable, baselines need to be established, and expected 
results need to be defined clearly from the start of the intervention. 

Interventions take time and need to occur at sufficiënt scale in order to have an impact, 
even at the micro-level. This means that evaluations must take into account a longer time 
frame, which deviates from CBI's current evaluation practice. Ideally, sufficiënt data should 
be provided to adequately measure the contribution of CBI’s activities within the 
perspective of a longer time frame. Effective monitoring requires all the necessary data to be 
in place. Ongoing information on (export) turnover by the beneficiaries is currently lacking. 
Submitting these data, and other relevant data, should be contractually arranged between 
CBI and beneficiaries. 

Enhance efficiency at the organisational and programme levels 

More supervision from MFA 

The agency model, which should have led CBI to deliver its services more efficiently, never 
materialised. The ex ante cost price model has not delivered the desired result from a 
fmancial perspective. MFA could revert back to the old system of ex post fmancing on the 
basis of delivered results. This approach will still generate the necessary flexibility to 
implement the programme. MFA could also be stricter with CBI in setting and justifying its 
cost prices in a situation of ex post fmancing. 
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Better cooperarion and coordination 

Although the ECP programme, as a five-year integrated export support package, is unique, 
only a relatively small number of the sample of assisted companies in the case study 
countries managed to increase their exports. The reason for this disappointing result is that 
the programme has not been able to deal with other constraints, some of which lie outside 
the sphere of influence of the current programme design. 

Since many of the programme’s components are also offered locally as standalone services, 
CBI could enhance the sustainability of its interventions by coaching such local service 
providers. Instead of CBI organising a trade fair, CBI could advise local service providers on 
how to (better) organise market entry activities, such as trade fair participation. 

CBI beneficiaries are used to working with a number of international partners, but good 
coordination between service providers is often lacking, which leads to a suboptimal 
division of tasks. Coordination between donors is a classic dilemma. It not only requires CBI 
to be (more) flexible in supplying the right intervention or part of a wider intervention, but 
should also lead to a better division of labour and regional presence at the PSD donor level 
in general. 

lnclude more local expertise 

In terms of programme expenditures, we see that the dominant share is concentrated in the 
hands of a small number of EU consultants. There are excellent local experts, who are often 
better at understanding the local context, that could also contribute to more efficiënt 
programme delivery and sustainable intervention results. It would make more sense to 
team up EU experts with local experts, which requires more of a supervisoty role from the 
former expert group. 
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ï.i Rationale and objectives 

The Centre for the Promotion of Imports from Developing Countries (CBI, the acronym of 
Centrum tot Bevordering van de Import uit ontwikkelingslanden ) is one of the Private Sector Develop- 
ment (PSD) programmes supported by Dutch development cooperation. These programmes 
are currently receiving a lot of attention because the ministry has made PSD policies a high 
priority. Itis therefore important to knowwhether, how, and forwhom they workbest. 

The establishment of CBI in 1971 should be seen in the context of UNCTAD II (New Delhi, 
1968), where the pursuit of developing countries to increase their market share in developed 
countries was discussed extensively (which was subsequently referred to as ‘Trade, not Aid’). 
CBI, an agency of the Dutch Ministry of Foreign Affairs (MFA), receives funding from the 
budget for official development assistance (ODA). Formally, the Regulation of the Minister 
of Finance on the establishment, organisation and operation of agencies ('Regeling 
agentschappen’ 2 ) requires agencies to be periodically evaluated - at least once every five 
years. 3 The last time CBI was externally evaluated was in 1999 by MFA, covering the period 
1990-1996. 4 CBI commissions external evaluations of its work on a regular basis. 

This evaluation of CBI intends to assess independently the results of the CBI programme 
and the extern to which it helps to achieve Dutch PSD policy objectives. 5 6 7 The current 
evaluation focuses on the programme’s effectiveness and efficiency, and addresses some 
issues that generate lessons that will improve programme delivery. 


1.2 Evaluation questions 

The ultimate goals of PSD policies are inclusive economie growth through greater 
investment, a less informal economy, decent jobs, increased productivity, and more 
competitive enterprises. This policy field is divided into five theme-based clusters, 'market 
development’ being one of them, to which the CBI programme is an important contributor. 
Hence, the central research question of the evaluation is: 

To what extern has CBI helped to achieve Dutch PSD objectives? 

Three OECD-DAC evaluation criteria were used to compile a list of nine evaluation 
questions, namely relevance, effectiveness and efficiency. 6,7 

2 ‘Regeling van de Minister van Financiën van 28 september 2012 over instelling, opzet en werking van 
agentschappen (Regeling Agentschappen)'. Staatscourant (Government Gazette) No. 20668; 15 October 2012. 

3 Article 7 of the regulation. 

4 IOB (i 999) Hulp door Handel; Evaluatie van het Centrum tot bevordering van de Import uit Ontwikkelingslanden 
(CBI), 1990-1996. IOB Evaluations 281. The Hague: Inspectie Ontwikkelingssamenwerking en 
Beleidsevaluatie. Octoberig99. 

5 See also chapter 2 and Annex 5 of this report. 

6 See: http://www.iob-evaluatie.nl/sites/iob-evaluatie.nl/fHes/ooo%20Evaluation%20Poli1y%20and%20 
Guidelines%20200g.pdf for a description of these criteria. 

7 The following link provides access to the Terms of Reference for the evaluation: http://www.government. 
nl/ministries/bz/organisational-structure/current-evaluations-foreign-affairs. 
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Relevance 

In order to determine the relevance of the CBI programme, its objectives and operations 
have been reviewed against insights from the scientific literature, as well as against the 
policies of MFA and export promotion policies of recipiënt governments. The aim is to 
assess in which countries or groups of countries and institutional settings CBI has a 
competitive advantage compared to other service providers. The following evaluation 
questions address the relevance of the programme: 

1) Was the programme (objectives, indicators and targets) consistent with the: 

• fmdings of the scientific literature on export promotion? 

• priorities set by MFA’s PSD policy? 

• priorities set by the home countries of the supported companies and BSODs? 

• fmdings from earlier/other evaluations? 

2) How has MFA operated with regard to formulating the assignment, controlling the 
budget, and informing CBI of changes in policy, as well as monitoring the results agreed 
upon with CBI? 

3) Could the objectives of the programme have been achieved through other means, e.g. 
direct support to export promotion agencies in target countries (either initiated by the 
private sector or through government)? 

Effectiveness 

Second, this evaluation sheds light on the effectiveness of CBI's services. Effectiveness is 
deflned here as the extent to which the outputs have contributed to the project’s expected 
results or objectives at the intermediate outcome level. For activities related to the private 
sector, effectiveness should also be judged by the programme’s additionality. Special 
attention has been given to issues which are less systematically focused on regular 
programme evaluations, such as attribution, additionality, and the sustainability of the 
interventions. The following questions guided this part of the evaluation: 

4) How were clients selected? What are the profiles of the selected beneficiaries? 

5) To what extent did participating companies increase exports to EU/EFTA as a result of 
CBI's interventions? Which types of company benefited most? What is the effect on 
employment as a result of the interventions? 

6) In addition to the direct effects, what kind of indirect effects (e.g. exports to other 
destinations than EU/EFTA) can be linked to CBI interventions? How did these indirect 
effects contribute to the programme objectives? 

7) To what extent did benefits (exports and jobs) continue after CBI’s interventions were 
completed? 
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8) To what extern did CBI collaborate with other development partners, and to what extent 
have benefidaries of the CBI intervention also been assisted by other trade promotion 
agencies, donors and stakeholders? 

Efficiency 

Efficiency concerns an intervention’s outputs, i.e. the direct results of the intervention 
achieved by means of the inputs. It measures the extent to which the outputs have been 
achieved as a result of the agreed-upon inputs, and whether the achieved outcomes could 
have been delivered for less money (cost-effectiveness). 

9) How do costs per service and per realised extra unit of export (added) value compare to 
that of other trade promoting organisations? 


1.3 Approach and methodology 

This evaluation is based on a mixed-method approach, using insights gained from 
qualitative evaluation methods (literature review, document analysis, interviews, surveys, 
four country case studies, and an external organisational benchmark), and combining this 
with quantitative methods where possible. 8 

Since the main products that underlie this evaluation report (i.e. country reports, survey 
and organisational benchmark) contain a detailed description of the methodology used, 
which can be read separately, this section provides a general description of the building 
blocks upon which evaluation outcomes have been based. 9 The relevance of the programme 
has been assessed by: 

• a review of literature on the role and impact of trade promotion agencies; 

• a short reconstruction of Dutch government policies and guidance that have set the 
framework for CBI to formulate and implement its development interventions; and 

• interviews with CBI staff, exporters, policy-makers, and other stakeholders. 

The effectiveness of the programme has been assessed by means of: 

• document analysis, evaluation reviews and other relevant documentation, including 
monitoring and cliënt relation management data; 10 

• semi-structured interviews with CBI staff, specifically with responsible managers for ECP 
interventions, and external consultants during the given period; 

• a telephone survey among beneficiaries, drop-outs and non-beneficiaries; 

• a telephone survey among importers; and 

8 Due to a lack of access to finandal data for all beneficiaries, it has not been possible to develop a 
rigorous quasi-experimental research design that compares intervention and control groups. 

5 The ToR of this IOB evaluation also has a section detailing the methodology. 

10 Information is based on the cliënt relation management System (SAGE) and monitoring data (HBAT). 
Since SAGE has been implemented during the course of the current evaluation time frame, CBI has 
retroactively completed the CRM database. 
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• case studies in two lower-middle-income countries (Egypt and India) and two upper- 
middle-income countries (Colombia and South Africa). 

Since it turned out to be impossible to obtain a credible dataset on the financials of 
CBI-assisted companies, a rigorous quasi-experimental research process, comparing 
intervention and control group, was not executed. Despite this, apart from programme 
beneficiaries, non-beneficiary companies (including drop-outs) have been surveyed as well. 
Since these ‘non-beneficiaries’ do not have the status of a ‘control group’ in statistical 
terms, we refer to them in the report as the 'reference group’. 11 

The exporter’s survey was held among 154 companies, including 118 beneficiaries, and 36 
non-beneficiaries (including drop-outs), covering a representative sample of the portfolio 
in 17 countries. A net response of 27% was attained for the beneficiary group. 

Finally, the efficiency of the programme was assessed against a benchmark of CBI’s 
organisational set-up and cost structure and compared to that of other trade promotion 
organisations. 


1.4 Reading guide 

Chapter 2 provides a short introduction to the Dutch government’s private sector 
development policies and the role that CBI plays in terms of achieving these goals. This 
chapter is followed by a description of CBI’s work procedures and institutional embedding 
(chapter 3). CBI's contribution to export development (chapter 4) is followed by an 
assessment of the efficiency of programme delivery (chapter 5). Chapter 6 discusses the 
relevance, effectiveness and efficiency of the Business Support Organisation Development 
programme. 


The reference group was compiled by asking CBI-assisted Business Support Organisations to suggest 
comparable companies from their files that are not CBI-assisted. CBI-assisted companies were asked to 
suggest names of comparable companies that were also not assisted by CBI. 
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2.i Introduction 

This chapter provides a short presentation of Dutch government policies and guidance 
(section 2.2) that have set the framework for CBI to formulate and implement its 
development interventions, i.e. CBI’s position within the framework of Dutch policies on 
market development (section 2.3). CBI's programme activities are described in section 2.4. 
A simplified theory of change and intervention logic sheds light on how CBI’s programme 
activities should contribute to the government’s wider aims (section 2.5). 


2.2 Market development policies within PSD 

Policies on private sector development by MFA are organised into five main intervention 
categories: infrastructure development, Financial engineering, improved market access, 
knowledge transfer, and interventions in the area of law and regulation. 12 

Regarding the Financial commitment in the ‘market development’ category with a gross 
expenditure of EUR 180 million out of a total of EUR 550 million (33%), CBI is by far the 
largest programme in this cluster (2005-2012). Other programmes include Netherlands 
senior experts (PUM), the Sustainable Trade Initiative (IDH), and the market development of 
Solidaridad, for example. Improved access to markets and improved product ranges are the 
two central pillars of the market development category. Whereas in the past emphasis was 
laid on access to the European market and the liberalisation of international trade through 
WTO negotiations under the Doha Development Round, this focus has been broadened to 
also make better use of South-South trade, 13 and pursue better production strategies. The 
assumption is that this will improve product quality, increase the number of value-added 
products, and enhance product diversification, which will broaden the export base of 
developing countries, making them depend less on the predominantly low value-added 
export of natural resources. The main outcome indicator of the market development 
category is increased domestic, regional, and international sales by companies in 
developing countries. 14 


* IOB has recently evaluated Dutch support to the development of the private sector in developing 
countries, implemented in the period 2005-2012. In search of focus and effsdiveness. IOB Evaluation 
no. 389, January 2014. See in particular Chapter 3: The Dutch policy for private sector development, 
pp. 5S-73. 

13 In 2013 the CBI received an extra assignment by the Ministry of Foreign Affairs to research possible 
interventions to enhance regional South-South trade and introducé corporate social responsibility. 

14 TK 2011-2012,32S05, no. 56. 
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2.3 CBI’s mission and strategy 

CBI's mission is to contribute to sustainable economie development in developing 
countries by increasing their exports. Therefore it is necessary to strengthen SMEs in 
developing countries, empowering them to cope with trade barriers and keep track of 
trends and market opportunities. By meeting stringent market demands, a growing demand 
for sustainably produced products should develop, leading to export growth, employment 
creation, and income generation. This should eventually result in poverty reduction and 
increased self-reliance. 

From 2005 onwards, international cooperation increasingly shifted the focus away from 
support projects that directly target beneficiaries on the micro-level (such as individual 
companies) to intensified efforts to achieve structural change in partner countries (support 
to business support organisations). Important reasons for shifting programmes towards 
building institutional capacities (meso-level) and addressing the policy environment 
(macro-level) were doubts, substantiated by research and practical experience, about the 
efficiency and sustainability of approaches that target the micro-level only. Most donor 
agencies, including CBI, follow a multi-level approach that simultaneously involves support 
measures for target groups, capacity development for institutional actors in their respective 
fields, and that also addresses structural bottlenecks in the policy arena, such as laws and 
regulations and sectoral promotion policies. 15 

CBI's Ieadership has not always agreed on its mission. There are basically two camps who 
have different ideas about the subject ofintervention. First, there are those who believe that 
interventions should focus on decreasing information asymmetiy, i.e. by exclusively 
focusing on the exchange and provision of market information. Others, however, state that 
companies and business support organisations need advice and counselling in order to 
develop their export capabilities. In their view, the intervention should go beyond simply 
providing market information and also include business development in general. 

The latter point of view has become the Standard operating procedure at CBI. This is 
illustrated by the ‘Working Instruction Export/Business Audit for ECP modules' (annex 6), 
where the company audit has become more of a diagnostic tooi that is used to lay out an 
intervention pathway to address these companies’ weaknesses, 16 instead of using the audit 
as a strict selection tooi for programme admission. 


15 Policy advice was just recently added to CBI’s portfolio, but no structural interventions have been 
defined in this area yet. 

16 This working procedure illustrates that every company is audited on various aspects by CBI, and the 
audit scores determine which intervention pathway is offered to the company. According to those who 
believe CBI’s main mission is to decrease information asymmetry, CBI should have been more selective 
in choosing its beneficiaries, only taking companies on board that are on the brink of exporting to the 
EU, since CBI is explicitly not meant to be or become a development organisation. 
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2.4 CBI’s programme activities 

CBI's multilevel intervention strategy aims to support individual companies as well as 
intermediary business support organisations (BSOs). In fact, a company might receive direct 
assistance through an export coaching programme (ECP), or benefit indirectly as the result 
ofCBI support to BSOs (see Figure 1). The backbone of CBI's interventions, also in terms of 
budget, are ECPs for individual companies and assistance to the BSOD programme. Two 
other main activities are the provision of market information and training (human resource 
development). Since these latter activities are more subservient to both ECP and BSOD 
programmes, the current evaluation predominantly focuses on both ECP and BSOD 
interventions that started and were fmalised between 2005 and the first quarter of 2013. 

Figure 1 CBI’s targeting strategy 
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Source: IOB based on CBI. 


Export coaching 

ECP programmes aim to strengthen the competitive capacity of SME producers/exporters in 
developing countries on European markets in order to achieve sustainable export growth. 
During the approximately fiveyears of an ECP intervention, selected companies are assisted 
in developing an improvement plan (based on an initial audit) and offered sector-specific 
technical assistance. This means export marketing training as well as market entry strategies 
(participation in international trade fairs and buying missions, for example). Participants 
receive individual support by means of on-site consultancy, coaching from a distance, 
market information, trade fair participation, and business-to-business activities. Depending 
on their specifk needs, companies may also receive technical assistance in the field of 
business development, certification, and product and production improvement. The phases 
and main activities deployed within the framework of an ECP are indicated below (Table 1). 
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Table 1 ECP programme phases and activities 

Phase (contractor) 

Activities 

1. Feasibility study 

- Country/sector selection 

2. Identification and pre-selection 

- Recruit participants 

- Identify missions and launch workshops 

- Pre-select applications 

3. Technical assistance (by external consultant) 

- Final selection of participants 

- Export auditing 

- Action plans and distance guidance 

4. Training (EXPRO) 

- Various seminars and workshops 

- Write export marketing plan 

5. Market entry 

- Trade fair participations 

- Company matchmaking 

6 . Market consolidation 

- Contact consolidation and follow-up 


Source: adapted from Triodos Facet (zoog). 


To be deemed a ‘competent exporter’ 17 by the CBI sector expert and CBI programme 
manager, a CBI-assisted company has to meet three sets of criteria. First, it should fulfil 
quality indicators at the result, effect and impact levels. 18 At the result level, the company 
indicates during the final evaluation of the EXPRO training that its skills and knowledge 
have improved as a result of having participated in the seminar. Second, the company 
translates its knowledge and insights of the EU and EFTA markets and export marketing into 
an export marketing plan. Finally, the company should have made relevant contacts in the 
EU/EFTA markets during the various market entry activities. 

At the effect and impact levels, the CBI quality indicators verify that the company did 
manage to increase its exports to the EU during the programme (effect). The total realised 
additional export turnover to the EU is referred to as the impact realised. 

The ECP portfolio accounted for in this evaluation totals EUR 24 million and represents 36% 
of the total expenditures on ECPs since 2005 (Figure 2). 


' 7 The word ‘competent’ exporter is debatable, as we have shown that for a substantial number of 

companies no financial data have been found in the programme database, which makes it impossible 
to condude whether these companies fulfïlled the quality criteria agreed upon between the dient and 
contractor. Moreover, as we saw with the companies for which financial data were displayed, we found 
that a substantial number did not achieve export turnover at all. Nevertheless, we will use this 
terminology throughout the evaluation. 

18 SeeAnnexg. 
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Figure 2 Breakdown ofECP Expenditures (CBI completed/deUvered companies) by country income 
group and sectors (2005 - March 2013, total EUR 24 mln) 

1 

upper-middle-income I 
lower-middle-income 
low-income economies I 

0 5 10 15 20 25 

amounts in EUR million 

■ agriculture, food and forestry ■ consumer products industrial products ■ services 
Source: CBI/SAGE, elaborated by IOB. 


From the above presented portfolio of completed ECPs, 3% of total expenditures (CBI 
delivered/completed companies) was realised in low-income countries, by far the largest 
share (90%) in lower-middle-income countries, and 7% in upper-middle-income countries. 
The budget breakdown by economie sector indicates that the main focus lies with industrial 
products. 19 

Business Support Organisation Development 

The BSOD programme aims to improve the business environment by enhancing the 
efficiency and effectiveness of the BSOs in their export development and promotion 
activities in the European market, and to improve the service delivery to exporters. Of the 56 
BSOD modules that have been implemented since 2005 and finalised by the end of 2012, 
evaluations of 22 modules implemented by or on behalf of CBI have been included in the 
research sample, representing 38% of the budget spent. 


' s Since CBI benefidaries are not consistently earmarked by main sector of economie activity, figures have 
been extrapolated. 
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2.5 CBI’s production 

The production values listed for the four main CBI services (dashed lines) show that ECPs are 
the main cost driver, covering almost 50% of the total production value (Figure 3). 20 


Figure 3 Production value ofCBl services 



Source: annual report s and production figures. 


Both BSOD modules and market intelligence products represent nearly EUR 26 million, 
whereas interventions related to training represent over EUR 35 million. 21 The production 
peaks are explained by the fact that ECP modules are launched in batches and take several 
years to complete. 22 


2.6 Theory ofchange and intervention logic 

A theory of change (ToC) explicitly shows how a programme or specific intervention is 
intended to function in a particular context, and how it will bring about the planned 
outputs and outcomes. It also explains which causal mechanisms are supposed to be 
triggered by an intervention, what the underlying assumptions for those causal 
mechanisms are, and what potential other development pathways could explain the results 
that have been realised. 


It should be noted that because the budgeting system changed in 2010, the production value since then 
has been based on an ex ante Standard tariff per delivered product. This value does not necessarily 
represent the costs of production. 

The HRD hiccup and ECP decrease in 2010 can be attributed to administrative issues. 

There is an additional explanation for the high ECP production peak in 2007, as this marks a change in 
the budgeting procedures from ex post product-based to ex ante module-based financing. 
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IOB recently reconstructed an MFA theory of change at the PSD policy level, which will be 
briefly touched upon in this section. 23 Better access to markets is one of the central 
objectives of the PSD policy. The malfunctioning of markets causes suboptimal terms of 
trade, as a result of which different groups are unable to benefit sufficiently from the 
opportunities presented by trade. The literature mentions two problems: (a) limited access 
to input or output markets, and (b) low or unbalanced competition in goods and services 
markets. A lack of market access hampers the development of a balanced private sector. This 
lack of access can have several causes: 

1. Entrepreneurs have poor access to the market due to a lack of infrastructure or the low 
quality of infrastructure, with high (prohibitive) transport costs as a result; 

2. Entrepreneurs have insufficiënt access to information about markets; and 

3. The dissemination of information on new production methods and techniques is 
inefficiënt or slow, as a result of malfunctioning public information services, for example. 

Especially for the poorer small-scale producers, high transaction costs, market risks, and 
limited information are important causes of poor access to (export) markets. 24 Problems 
arising from the structure and functioning of markets in developing countries include: 

1) a lack of networks between producers, traders, and processors, which increases the 
costs involved in commercial transactions between companies; 

2) small-scale producers cannot achieve sufficiënt volume in the market and are therefore 
at the mercy of middlemen; 

3) a lack of knowledge and information on prevailing market conditions; this concerns 
both prices and quality requirements in domestic and international markets; 

4) a limited number of commercial parties that together determine the price and block 
new parties from entering the market (monopoly or oligopoly formation); 

5) trade barriers and limited interaction with the world distort domestic price ratios; and 

6) low integration of local, regional, and national markets, potentially causing large price 
fluctuations and disproportionate margins. 

CBI’s theory of change 

CBI has never explicated a theory of change at the programme level, 25 though it did start to 
base each of its interventions on logical framework analyses. Therefore, we have 
reconstructed a simplified ECP-oriented theoty of change for this evaluation. 

The simplified theory of change assumes that initially any company could be a flourishing 
exporter, except that certain constraints might be prohibiting this. It further assumes that if 
these constraints are - to some extent - removed, by ECPs or by other external or internal 
factors, the company will be more likely to increase its export turnover (Figure 4). 

23 See also Annex 8. 

24 SDC/DFID (2008). A Synthesis of the Making Markets Work for the Poor Approach (MqP). Bern. 

25 CBI prepared a ToC ahead of this evaluation, but this document is more of an elaborate description of 
its four main products than a ToC. In response, IOB organised a workshop with the CBI programme 
managers, management and M&E staff, to reconstruct an intervention logic. 
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Figure 4 Schematic overview oftheory ofchange 
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Source: IOB. 


Potential export constraints are summarised and categorised in Table 2. Exports materialise 
when a company’s supply meets demand in the foreign market. Therefore, export 
constraints on the demand and supply side are related. 


Table 2 Trade-related constraints 


Supply-side constraints 


Context-related constraints 


Demand-side constraints 


Cost price level 
Design quality 
Product quality 
Packaging 

Parts and raw material 
supply 

Internal export 
organisation/capacity 

Production capacity 


Trade policy restrictions 
Exchange rate 
Currency convertibility 
Import restrictions 
Financing (of export transactions) 

Shipping delays and costs 


Market entry 

Customer needs and preferences 
Demand changes and trends 
Market size and growth 

Market access (tariff and 
non-tariff barriers) 

Insufficiënt demand 

Communication with foreign 
markets/buyers 


Source: IOB. 


For instance, the packaging of a product (a supply-side factor) can only be considered to be 
constraining export performance when it does not meet the customer’s needs and 
preferences on the demand side. Supply-side constraints are (mostly) under the direct 
control of companies. For instance, the cost price of a product is very closely related to the 
organisation of production (due to, for instance, organisational decisions on economies of 
scale and the level of mechanisation), but also to other factors outside the company’s 
sphere of influence, such as the labour standards the company has to adhere to. 






CBI’s role in private sector development 


Demand-side barriers, on the other hand, cannot be reduced by a company. However, a 
company can potentially overcome these barriers by adapting itself or by committing more 
resources. For instance, a company cannot reduce the certification requirements for 
accessing a foreign market, but it can get its products certified; and although market entry 
in Europe is difficult and costly, a company can decide to commit resources to participate in 
a trade fair. 

Context-related export constraints are different in this respect because they are considered 
more difficult to overcome. Context-related factors include, for instance, the exchange rate 
or import tariffs. 

Simplifïed intervention logic 

Other than a theory of change, the intervention logic specifically describes the internal logic 
of the programme intervention that ultimately should lead to the projected results at the 
output, outcome, and impact levels. The logical frameworks supporting CBI interventions 
correspond to the intervention logic. Annex 7 reconstructs the intervention logic of both 
ECP and BSOD programmes, which form the core object of evaluation. 26 

Limitations of the evaluation 

CBI's tradition of programme evaluation has improved markedly over the years. 

By introducing a programmatic approach in 2011, 27 CBI has included a result chain in the 
measurement and evaluation plan. 28 The ‘old style’ programmes, which are the subject of 
the current evaluation, do not work according to this result chain and instead used to 
monitor so-called ‘quality indicators’ listed for the different CBI products. 29 Quality 
indicators and standards were set in order to enable CBI to steer on the basis of the quality 
of its activities. 


26 The reconstructed intervention logic finds its base in a joint session with the CBI programme 
management sta ff. 

27 Essentially, the new approach that was introduced in 2011 should be more balanced and integrated as a 
result of a reinforcement of services, which are to be more focused on addressing capacities and 
bottlenecks in the countries of operation, instead of market demand only. 

28 In January 2013, MFA approved CBI's new ‘Measurement b Evaluation Plan'. This plan is based on the 
principles of‘Protocol Resuttaatbereiking en Evalueerbaarheid in PSD zon'. CBI’s new programme management 
System is aligned with the DCED Standard. After a few years, it should deliverdata on outcome and impact 
levels. Currently, the development of a real-time evaluation method is underway (PRIME). 

29 Annex 9 gives an overview of the key performance indicators and quality indicators per intervention 
area and level, applied by CBI in the period 2006 up to December 2012. It must be mentioned, though, 
that the suggested breach in monitoring and evaluation does not mean - especially for the ECPs - that 
the ‘old style’ indicators have become completely obsolete. 
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A set of key performance indicators per intervention area, applicable for all ‘old style’ 
products, was adopted in 2006. 30 This allowed CBI to operate in a more result-oriented and 
flexible way, more in keeping with its status as an agency responsible for its own income 
and expenses. 

Performance indicators were developed to serve two goals, namely: (1) project monitoring 
(content) and (2) project administrative purposes (accountability). A set of quantitative and 
qualitative indicators made it possible to objectively verify the number of completed units, 
while enabling fmancial auditors to verify whether CBI has the right to claim settlement of 
the received advance payments. 

From a practical point of view, though, the programme management system did not-and 
still does not - meet the current Standard of result measurement and evaluation. Despite a 
comprehensive dataset that was specially prepared by CBI, IOB’s questions challenged the 
administrative and fmancial reporting system. 

No data is being specifically collected to establish a baseline or end-line situation for CBI’s 
interventions - at least not systematically. Moreover, evaluation outcomes are 
predominantly based on self-perception, measuring cliënt satisfaction is not an integral 
part of the evaluations, and non-response analysis is absent. There is an obvious lack of 
discriminative performance evaluation data. 31 Moreover, from a methodological point of 
view, the evaluations are also of limited reliability, as they do not systematically address the 
issues of attribution, additionality, and sustainability. 32 


30 Existing products were subdivided into modules allowing for more flexible programme delivery. 

31 Some examples include the objections that have been raised in several evaluations about the limited 
information of the EVALIN evaluation tooi, which is based on forms filled out by course participants, 
and interviews right after the interventions took place, instead of some time after completion of the 
training. Similarly, some evaluators questioned whether CBI should use a longer time frame to measure 
additional export turnover at the company level. 

32 Since CBI does not stipulate the reason why companies drop out of the programme, for example, 
current monitoring data do not allow one to measure the potential upward bias of the programme 
effect (selective attrition). 
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2.7 Findings 

• The focus of international cooperation has shifted away from support directly targeting 
companies to support that aims to achieve structural changes in partner countries. Most 
donor agencies, including CBI, now claim to follow such a multi-level approach. In 
practice, however, CBI’s interventions still focus more on the individual company level. 

• CBI's leadership has not always agreed about what CBI’s mission is. While some believe 
that CBI should focus on decreasing (trade) information asymmetiy, others are 
convinced that beneficiaries need general business development, which requires a 
broader set of interventions. The latter reasoning has become the guiding principle of 
the CBI programme. 

• CBI achieved just 3% of total expenditures in low-income countries, 90% in lower- 
middle-income countries, and 7% in upper-middle-income countries. This kind of 
budget allocation deviates from the priorities set by MFA, one of which is to target 
low-income countries more. As it takes some time to phase out programmes, 
expenditures have become more aligned with MFA demands, though it remains difficult 
to scale up in lower-income economies. 

• CBI does not have an explicit theory of change that explains how the programme is 
meant to function in a particular context or how it will bring about the intended 
outputs and outcomes. Such a ToC had to be reconstructed for the purpose of this 
evaluation. However, CBI’s interventions are based on an intervention logic that reflects 
the reconstructed ToC. 

• CBI's programme management system does not meet the current Standard of result 
measurement and evaluation. This makes it difficult to effectively monitor, steer, and 
evaluate the programme. 
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CBI’s working procedures and institutional embedding 


3.1 Introduction 

CBI's work procedures do influence the final results of its activities. This chapter examines 
to what extern work procedures are consistent with the interests and needs of the 
beneficiaries, within the changed institutional context in which the organisation operates. 
These procedures are related both to the selection of countries and economie sectors and 
sub-sectors (Section 3.2), as well as the selection procedures followed to admit a company 
into the programme (Section 3.3). Another factor that likely influenced the results of CBI’s 
activities can be traced back to CBI’s institutional setting, having been transformed from a 
governmental department into an independent agency (Section 3.4). 


3.2 Institutional context and country/sector selection 

Stimulating national exports by offering information, finance, and technical assistance to 
individual firms through export promoting organisations (EPOs) has become an 
increasingly popular government intervention. The number of EPOs representing national 
interests has tripled worldwide over the last two decades (Lederman et al., 2010). The 
literature tells us that the types of service offered by these EPOs seem - at least on the 
surface - quite similar across countries. 

One could argue that the increase in EPOs has opened up CBI’s market for delivering 
support to newly established EPOs, helping them to become more responsive and 
professional. On the other hand, one could also say that the increase in the number of 
initiatives promoting export would have induced CBI to become more selective regarding 
initiatives to support or gaps to fill up. In other words, it raises the question which 
considerations played a role to persuade CBI to become active in certain countries and 
economie sectors and sub-sectors. 

According to CBI the choice to become active in specific market sectors in developing 
countries is based on the following criteria: 

• import potential in EU/EFTA markets; 

• willingness among potential buyers in EU/EFTA markets to establish relations with 
exporters in developing countries; 

• adequate export potential in developing countries; 

• no insurmountable trade barriers; 

• products have added value (no traditional bulk commodities); and 

• contribution to export diversification of developing countries. 

It is important that the production capacity has sufficiënt potential to be competitive to 
conquer, maintain, orexpand a market share on the European market. Furthermore, CBI 
also takes into account the business climate and social policies of these countries. Countries 
with a per capita gross national income equal to ’upper-middle-income’ and higher, 
including China, are excluded. Based on the available portfolio during the period of 
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evaluation, 90% of total expenditures was spent in lower-middle-income countries, while 
3% and 7% was spent in low- and high-income countries, respectively. 

Based on feasibility studies, CBI selects countries and economie (sub-)sectors for the 
programme. However, decisions to start an intervention are based on import predictions 
for the EU industry in general rather than actual market needs (as defined by importers). 

A review of country and economie sector selection procedures within the ECP programme 
shows that these sometimes deviate from the guiding principles and procedures. For 
example, a comparison of the CBI country list and the countries of operation that were 
actually chosen, shows that ‘preferred’ countries were not the only ones to be selected but 
also countries marked as ‘doubtful’ or ‘not selected’, without explicitly explaining such a 
choice. 33 
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33 Selection is based on the same scoringformat as for the feasibility criteria, whereby the criteria ‘market 
prospects’ and ‘supply potential’ are comprehensive and - methodologically speaking - soundly 
underpinned. The remaining sections - ‘assistance’, ‘design of the ECP’ and ‘policy’ - receive 
considerably less attention in the ECP feasibility reports. For many indicators in the latter sections, itwas 
not possible to identify a rationale for the scores underlying the feasibility criteria tables. 
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Box i Changes in operational countiy list 


Policy changes introduced by succeeding ministers have had implications for the 
operational country list where PSD interventions could be programmed. According 
to the 2005 annual plan, CBI was able to operate in 35 countries, including 23 
bilateral partners. Western Balkan countries were added to the list in 2005, and in 
2006 CBI was assigned to this region. During the following years, the operational 
country list increased to 47 (including Armenia, Georgia and Moldova, and Burundi, 
while Rwanda and Afghanistan are being considered). 

The introduction of a revised - and strongly reduced - operational country list by 
MFA in 2008, faced broad resistance and triggered the founding of the ‘PSD 
platform', a round table of PSD implementing bodies and the ministry, initiated by 
CBI. In a memo from January 2009, the ministry requested PSD organisations to 
establish better contact with the least-developed partner countries, and 
simultaneously slowly phase out activities in (upper) middle-income countries. 

In response to the increase in the number of countries with limited possibilities for 
export to the EU (i.e. Democratie Republic of Congo, Yemen, Mongolia, the 
Palestinian Authority, and Sudan), CBI proposed to expand its assignment to 
include enhancing regional South-South trade, as a preliminary for increased EU 
exports. This led to a revised country listing in 2010 encompassing the following 
country classifications: (a) nine fragile States, seventeen MDG countries and two 
least-developed countries (LDCs); (b) seven ‘broad relation’ countries in which CBI 
could start programmes up to the year 2014; and (c) fïve exit countries where CBI 
should have phased out its activities by 2013, seven promising economie partner 
countries and fïve other non-partner countries. 

CBI’s 2010 annual plan mentions a focus on development partner countries as well 
as LDCs, totalling 70% of the budget spent in 2014, with the remaining 30% to be 
spent in ‘broad relation’ countries. This meant a reduction in spending in ‘phasing 
out’ countries from 52% in 2009 to 0% in 2014. In 2011, MFA requested CBI to 
report in the future about its activities according to the following country groups: 
fïfteen focus countries, three transition countries, twenty-nine other PSD countries 
and seven exit countries - totalling 54 countries. In its 2013 annual plan CBI refers 
to expenditure percentages in the following country categories, earmarked for 
2017: focus countries (40%), transition countries (20%) and other PSD countries 
(40%). The current selection of 13 ECPs evaluated in this report covering the period 
2005-2012 reveal the following expense patterns in 26 countries in which CBI has 
been operating: exit countries (49%), focus countries (4%), transition countries 
(16%) and other PSD countries (31%). 
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ECP expenditure by size by country classification (13 ECPs, 2005-2012) 

Essentially, ECPs were not run exclusively in one or just a few countries, but open to a 
‘global’ clientele. The number of countries participating in an ECP ranged from one to 
fourteen. In total, the thirteen ECPs under review represent 410 ‘competent exporters’ from 
twenty-six countries, i.e. on average fifteen companies per country have been a beneficiary 
in one of these thirteen programmes. 34 Such a dispersed allocation of resources, already 
ex ante, carries with it the inherent risk of becoming too diffuse to be able to make a 
reasonable contribution to poverty reduction, which remains one of MFA’s central policy 
aims. 

The document review indicates that CBI is generally aware of the importance of contextual 
issues as potential causes for the observed intervention results. In its annual reviews, CBI 
elucidates on contextual developments that influenced production. It mentions the 
following issues, for example: 

• phasing out of the Multi Fibre Agreement, which hampers textile producers; 

• a review of the EU Novel Food regulation, which is considered to be a serious barrier for 
SMEs in developing countries that are active in this sector; 

• product energy use as an additional criterion for CE certification; 

• Forest Stewardship Council (FSC) labelling, requiring a substantial investment which 
cannot easily be compensated by gains; 

• EUREP/GAP certification costs, which are an insurmountable barrier for companies; 

• difficulties that companies face adhering to conformity assessments, which are related 
to standards and regulations; 

• exporters from developing countries seem to be losing ground because of e-business; 
and 

• CSR aims to stimulate transparency in markets, and requires a value-chain-based 
approach. 

A critical reflection on the implications of the developments mentioned above for the 
choice of country-sector combinations did take place. CBI introduced an ‘Access Guide’ (part 
of the market information product), providing information on non-tariff barriers. In 
response to the phasing out of the Multi Fibre Agreement, CBI initiated a short-track ECP on 
clothing (2003-2005) in India, Sri Lanka and Bangladesh. With regard to FSC labelling, 
clinics about FSC certification were organised within the 'Timber and Timber-related 
Products’ ECP (2000-2009). In the Netherlands, a stakeholder meeting was organised to 
discuss partnerships. Moreover, CBI introduced the 'Fresh Fruits and Vegetables' ECP 
(2002-2007), which had been specially started to support companies in low-income 
economies that risked losing their market position as a result of EUREP/GAP requirements. 


Since programme take-up is essentially globally focused, there are examples of ECPs with a dominant 
country clientele. See, for example, ECP 1034 (‘Pipe and process equipment’), accommodating 
predominantly benefïciaries from India and Pakistan, or ECP 1035 (‘Outerwear Macedonia’), being the 
only one of the 13 ECPs investigated that had benefïciaries from one country (Macedonia). 


34 
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Despite the above, not all countries will necessarily reap the benefits of enhanced trade 
openness, according to the literature review. Trade particularly seems to benefit more 
advanced economies with a good business climate (Winters and Masters, 2013). In this 
respect, flexible labour markets, the ease of starting a business and well-established 
property rights seem to be especially important (Freund and Bolaky, 2008). The idea that an 
enabling environment does matter, as suggested in the literature and also reflected in CBI’s 
selection criteria, raises the question ofwhat other information on the enabling trade 
environment could have been used ex ante to guide CBI’s country choice. 

To cite an example, ‘The Enabling Trade Index’ (ETI) 35 provides insightinto the fundamental 
attributes that govern a country’s ability to benefit from trade. ETI is a collection of 
individual indicators compiled in a single index on the basis of an underlying framework 
that captures the following trade-enabling dimensions: 

• market access, which measures the extern and complexity of a country’s tariff regime, as 
well as tariff barriers faced and preferences enjoyed by a country’s exporters in foreign 
markets; 36 

• border administration, which assesses the quality, transparency and efficiency of a 
country’s border administration; 37 

• infrastructure, which assesses the availability and quality of a country’s transport 
infrastructure, associated services, and communication infrastructure, all of which are 
necessary to facilitate the movement of goods within the country and across the border; 
and 

• operating environment, which measures the quality of key institutional factors 
impacting the business of importers and exporters operating in a country. 

When taking into account the share of companies that left the programme per country, i.e. 
the drop-out rate, these countries also seem to be the ones that face the highest barriers in 
terms of international market access. This is not to claim causality up front, 38 but illustrates 
that CBI could have enriched programme information and feasibility studies with publicly 
available information in order to reach a more well-reasoned operational country choice, 
and to create more valuable information for the steering of the programme. 


35 The Global Enabling Trade Reports, World Economie Forum. 

36 The pillar assesses tariff barriers faced by a country’s exporters in destination markets. It includes the 
average tariffs faced by the country as well as the margin of preference in destination markets 
negotiated through bilateral or regional trade agreements or granted in the form of trade preferences. 

37 The pillar assesses the efficiency and transparency of border administration. More specifïcally, it 
captures the efficiency, transparency, and costs associated with importing and exporting goods. 

38 The real-life picture is more ambiguous than the comparison suggests, as for example, not all 
companies experience international market access barriers equally. 
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According to the assessment of the ETI country rankings, it would have been less likely for a 
choice to fall on the following countries at the time of the start of the interventions (listed 
in order according to the amount of budget spent): Colombia, Bolivia, Uganda, El Salvador, 
Honduras, Tanzania, and Bangladesh. 39 

A preliminary conclusion, based on information from CBI’s programme database indicates 
that countries where the intended results were achieved are the ones that also score 
favourably on the ETI (in alphabetic order): Egypt, India, Jordan, Macedonia, Pakistan, Peru, 
Philippines, Serbia, and Tunisia. 

Countries with neutral or good perspectives for international market access where export 
potential has not been sufficiently realised, if at all, include: Ghana, Indonesia, Kenya, 
South Africa and Sri Lanka. 


3.3 Company intake and addressing its needs 

A specific characteristic of ECPs is the selection ofweaker and stronger companies under the 
assumption that the stronger companies will serve as a benchmark for the less strong ones 
(a 'healthy mix'). In addition, it is assumed that companies from countries with a weaker 
supplier profile or image could benefit from improved visibility and the reputation of 
stronger countries in the programme, for example by joining forces at a trade fair. 

When addressing company needs, the theory of change identifies the most important 
export barriers for companies from developing countries. By comparing these to the 
barriers addressed by ECPs, one can assess whether the assisted companies have found the 
ECPs to be meaningful. 

An illustrative example taken from one of the country case studies lists the biggest export 
obstacles ranked by scores according to 'The Global Enabling Trade Report’ 40 , and the 
combined scores of thirteen South African case study companies (Table 3). Moreover, the 
export obstacles mentioned by the interviewed trade promotion organisations have also 
been referred to in the table. The last column indicates whether the export barriers are in 
fact addressed by ECPs. By comparing the perceived barriers to export with the activities 
delivered within each ECP, one can judge ex ante the relevance of ECPs. The judgement as to 
whether, and to what extern, ECPs have actually removed these export barriers for ECP- 
assisted companies is part of the contribution analysis which follows in the next chapter. 
When trying to enter foreign markets, exporters face constraints related to market supply 
and demand, as well as the wider economie context within which companies operate. Each 
of the export barriers refers to one of these dimensions, as indicated in the third column. 


35 This assessment goes beyond the political wish of the Ministry of Foreign Affairs for the CBI to become 
more active in least-developed economies. See Annex io for a visualisation of the ETI rankings for each 
of the countries in which CBI has been running ECPs. 

40 The World Economie Forum, 2014. 
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According to the World Economie Forum (WEF), South Africa faces major export constraints 
while lacking access to imported inputs at competitive prices, which increases export prices, 
especially since the currency exchange rate has worsened. 41 A highly complex tariff structure 
for imports and the inefficiency of border administration procedures to import and export 
goods are also considered to be the cause of major problems. Half of the top-ten ranked 
export barriers are related to contextual issues. 

The export barriers perceived by the case study companies partly overlap with those 
mentioned in the WEF report (the First 12 obstacles listed in Table 3), but they also indicate 
that other issues are at play, such as a lack of internal export organisation (capacity), dealing 
with demand changes and trends, and communication with foreign buyers. 

Many of the constraints mentioned both by the WEF report and case study companies are 
also in line with export obstacles identified by trade promotion organisations (TPOs). 42 
Triangulating these sources helps us to identify the most important export constraints faced 
by South African companies: 

• no competitive pricing due to high cost of inputs (e.g. raw materials, electricity, labour); 

• problems identifying potential buyers; 

• tariff (import duties) and non-tariff barriers (technical requirements and standards); 

• limited Financial resources to overcome market entry costs (especially common among 
small companies); 

• high transport costs (and shipping delays), especially to Europe; and 

• difFiculties in meeting buyers’ quality/quantity requirements. 

This example shows that ECP interventions focus on two of the six major export constraints, 
namely: (1) the identiFication of potential markets and buyers, and (2) difFiculties in meeting 
buyers’ quality/quantity requirements. In addition, much effort goes into dealing with 
constraints that were considered less important (e.g. training on legal issues, contracts, and 
communication with foreign buyers). However, it should be noted that many of the 
important unaddressed constraints are context-related, and hence fall outside the sphere of 
influence of ECPs and export development programmes in general. 

What does this mean for the relevance of ECPs for assisted companies in South Africa and 
elsewhere? While ECPs are actively identifying buyers and markets, which is considered to 
be the most important export constraint among assisted companies, there are many other 
important factors that are constraining these companies that are not being addressed (e.g. 
high input prices). This implies that there was ex ante a considerable risk that the effort of 
ECPs, even if highly successful in bringing companies in contact with buyers, would not 
increase export turnover when these other constraints turn out to be binding. 
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41 These constraints are not from recent years, but were also apparent at the time of ECP implementation. 

42 The abbreviations TPOs and BSOs are interchangeable. 
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Table 3 Export barriers 



Obstacle 

Dimension 

WEF rank 42 

Case study 
companies 

TPOs 

ECP 

1 

High cost or delays caused by 
domestic transportation 

context 

X 




2 

Access to imported inputs at 
competitive prices/price level 

supply 

X 

X 

X 

(X) 

3 

Identifying potential markets 
and buyers 

demand 

X 

X 

X 

X 

4 

Tariff barriers abroad (e.g. 
import duties in target markets) 

demand 

X 

X 

X 


5 

Inadequate production 
technology and skills 

supply 

X 



(X) 

6 

Burdensome procedures at 
foreign borders 

context 

X 




7 

High cost or delays caused by 
international transport 

context 

X 

X 

X 


8 

Access to trade fïnance 

context 

X 

X 

X 


9 

Corruption at foreign borders 

context 

X 




10 

Difficulties in meeting buyers’ 
quality/quantity requirements 

supply 

X 

X 


X 

11 

Technical requirements and 
standards abroad 

demand 

X 

X 


(X) 

12 

Rules of origin requirements 
abroad 

demand 

X 


X 


13 

Limited knowledge on legal 
issues and contracts 

context 



X 

X 

14 

Currency volatility 

context 



X 


15 

Internal export organisation 

supply 


X 


X 

16 

Demand changes and trends 

demand 


X 


X 

17 

Communication with foreign 
markets/buyers 

demand 


X 


X 

18 

Design quality 

supply 


X 


X 

19 

Production Capacity 

supply 


X 


(X) 

20 

Packaging 

supply 


X 




(X) = ECP partly contributing; 

Bold = main obstacles ECPs address. 
Sources: World Economie Forum, IOB. 


43 Ranking from highest to lowest obstacle. 
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3.4 Institutional arrangement 

CBI was established in 1971 as the first international ‘import promotion organisation’ to 
develop non-commercial policy to increase imports from developing countries to Europe. 
Although the minister of development cooperation in 1997 indicated that he was satisfied 
with how CBI was functioning, it was decided to investigate whether CBI would function 
even more effectively if it was to be placed further away from the core department. A study 
decided that the preferred option was its transformation into an agency (see Box 2).™ On 
1 January 1998, CBI was spun off into an agency with its own income and expenses regime 
('Baten Lasten Dienst-model’ in Dutch). 

Box 2 R egulation departmental agencies, 2007 

Departmental agencies have been operational in the Netherlands since 1994. They 
can be defined as government organisations with their own income and expense 
regime, which aim to achieve greater effectiveness in policy implementation. In 
2011, the ‘B aten Lasten Dienst (BLD)’ scheme was evaluated by the Ministry of Finance. 

Departmental agencies are characterised by a results-oriented management 
model, supported by a cost-benefit administration. It is a form of internal 
autonomy in which the hierarchical subordination to the minister remains fully 
maintained. This model is based on two general assumptions: (1) the agency can 
similarly be controlled as a company in the private sector; and (2) the agency is not 
out to create economie value while serving the general public interest. The BLD 
model has the following specifïc characteristics: 

t. results-oriented management model: 

a. three distinct roles (contractor, dient and owner); 

b. output management by the dient; 

c. presence of conflict of interest/negotiation game; 

2. efficiency/measurability: 

a. handling a full cost model; 

b. presence of homogeneous measurable products and services; 

c. monitoring efficiency (using indicators); and 

3. different fïnancial regime: 

a. the presence of investment and depreciation, use of loan facility; 

b. building of equity; 

c. use of accrual accounting. 

Source: Regulation Departmental Agencies. MinistryofFinance, 2007 . 


« The reason for introducing agencies in the Netherlands was to lower the cost of services, without 
compromising on quality. The ministry and CBI have agreed on a 'results-based control model’ (price x 
quantity) per product or service. The indicators set by CBI to monitor its performance (key performance 
indicators) actually serve two goals: (t) project monitoring (content), and (2) project administrative 
purposes (ex ante price setting of services). 
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3.4.1 Result-oriented management model 

The contribution of CBI to the implementation of market development policies is the 
subject of consultation between MFA and CBI. The deputy Secretaty-General of the MFA acts 
as the ‘owner’ of CBI, while the MFA’s deputy Director-General of International Cooperation 
(DGIS) acts as the cliënt 45 on behalf of the Minister for Foreign Trade and Development 
Cooperation. The mandate that CBI, the contractor, should contribute to poverty reduction 
by strengthening the competitiveness of companies from developing countries, i.e. by 
improving internal business conditions, was reconfirmed by the cliënt in September 2009. 
CBI submits a multi-annual offer to MFA in which it presents its agenda. Within this general 
framework, the deputy DGIS issues annual contracts to CBI for products to be delivered on 
the basis of a tender issued by CBI at already approved tariffs. Being an agency implies that 
steering from the ministry focuses on products (quantity/price) and to a lesser extern on the 
policy impact and quality. 

Supervision 

According to the assumptions underlying the agency model, MFA's central Financial 
department (FEZ) 46 acts as advisor to the deputy Secretary-General, representing his or her 
interests. The control unit (CU) should focus on assessing the effectiveness of business 
processes at CBI, but also make sure that CBI’s cost rate applies. However, document review 
and interviews indicate that the CU is primarily involved in advising the deputy DGIS about 
the plausibility of the offer submitted by CBI, assessing whether the quoted rates are not too 
high. The latter should be the role of the deputy DGIS, supported by its controller. 

Several MFA departments have been in charge of steering CBI. 47 From interviews held with 
current and former policy-makers, controllers and employees of the Central Government 
Audit Service (ADR), and supported by detailed document analysis, one could generally state 
that despite a great deal of effort, commitment, and good intentions, there has been 
insufficiënt knowledge and operational know-how from within the ministry on how to 
manage and steer an agency that is supposed to behave as a private entity. 

The available files do not contain good descriptions of the different specific roles for the 
owner and cliënt, according to some interviewees, and this has even led to a situation in 
which the interpretation of the roles between cliënt and owner were reversed. While the 
interpretation of the assignment and the formulation of the results (the 'how to do’ 
question) were discussed with the owner, the cliënt focused primarily on more 
institutional-related topics (the ‘what’ question), suggesting that they interpreted their 
roles in a different way. 


45 The CBI now hasdefacto one cliënt, namely the deputy DGIS. Duringthe period underevaluation, CBI foresaw 
other potential clients over time, such as the European Union, Germany, Ireland, and the World Bank. 

46 The Dutch acronym FEZ stands for Financieel Economische Zaken. 

47 From the Social Development Department to the Sustainable Economie Development Department. 
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Specifically, since MFA began administering an agency ofwhich the operational and 
financial control system differs completely from that of regular cash and obligations 
Systems, 48 it can be concluded that the quality of MFA's administrating of CBI has depended 
strongly on the interest and knowledge, as well as available time and priority setting, of the 
civil servants in charge, both at the levels of strategy and implementation. Therefore it has 
been very difficult to administer a proper result-oriented framework that asks CBI the right 
questions. 

Agency model requires the owner, dient and contractor to play different roles 

As in any steering relationship between owner, dient and contractor there is a degree of 
information asymmetry: contractors have more information at their disposal than clients 
and owners. The mechanism of conflict of interest between the dient, owner, and 
contractor never materialised in the case of steering CBI. The documents reviewed and the 
interviews suggest that the negotiation game was never really fully played. Instead, the 
relation between owner, dient, and contractor can be characterised by a commonly shared 
interest or risk. 

Moreover, CBI, as the only service provider, had the opportunity to monopolise. In such 
cases, transparency must be introduced by benchmarking with comparable providers, from 
the private sector or not. However, for specific public goods there are hardly any alternatives 
that can serve as a reference to keep the pricing of services transparent. This prevents any 
real bargaining from happening, which is illustrated by the following example. 

Annually, CBI prepares a quotation for the deputy DGIS and, based on the offer, expects to 
engage in a business negotiation with the dient regarding the amount, price and quality of 
the services. In these types of bargaining situations it makes sense for the contractor to 
maximise the initial offer, thus creating room for manoeuvre. Since this kind of negotiation 
process rarely occurred between CBI and MFA, one could say that MFA did not push for 
efficiënt service delivery by CBI up front. 49 

Another intervening factor is the difference in accounting systems used by CBI and MFA, 
which are not compatible. MFA reserves approximately EUR 25 million of its budget 
annually for CBI, and administers a model that encourages cash flow instead of cost price 
reduction. If CBI were to reduce the cost of production, while MFA enforced a predictable 
annual cash flow, then CBI would have to increase production. This suggests that there is 
not an MFA-induced incentive for CBI to loweritscosts. Orvice versa: if the fundinghy the 
dient is solely based on the principle of price times quantity, then there is an inherent 
interest on the contractor’s side to maximise quantity during the year, without necessarily 
contributing to social welfare. 
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48 In Dutch: ‘regulier kas- en verplichtingen stelsel'. 

49 See also the report of the Accounting Unit on the cost prices offered to MFA by CBI. 
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3.4.2 Lack of steering by MFA 

Until 2010, MFA gave CBI only limited attention in policy, and the relationship between CBI 
and its principal was not particularly dynamic. A few parliamentary questions were raised 
about alleged subsidies that CBI left for competitors of Dutch companies in developing 
countries, thus destabilising a level playing field . 50 This lack of steering by its principal gave 
CBI room for manoeuvre, and it launched initiatives that were often accepted by the 
principal without much comment. Several times, CBI also tried to expand its mission . 51 

Since 2010 in particular, both MFA and CBI realised that the time had come to change the 
attitude that had crept in afteryears of sustained laissez-faire. Consequently, MFA became 
stricter with the agency. It was mutually agreed that CBI had been operating for too long in 
splendid isolation . 52 MFA asked CBI to introducé initiatives that contribute to the 
implementation of policies set in the field of private sector development. It required a more 
focused mission and goal-setting, in which CBI was challenged by MFA to establish a more 
tangible and transparent link between export promotion, sustainable development, and 
poverty reduction. In the ‘PSD protocol on evaluation’ it was agreed that all impact 
evaluations should measure net income (exports in the case of CBI), as well as net 
additional jobs . 53 MFA explicitly requested CBI to establish better connections with the 
Dutch (least-developed) partner countries, while stepping back its activities in (upper-) 
middle-income countries, and to address SMEs. 

Under the State Secretary for European affairs (2010-2012), an interest in supporting 
economie sectors was developed, promoting the self-reliance of developing countries. 
Policy changes focused less attention for aid, while stimulating more trade and investment, 
and promoting mutual beneficial relationships (also with a focus on strengthening the role 
of the Dutch private sector in development policy). Furthermore, the previously announced 
reduction in the number of countries in which the so-called private sector development 
tools could be implemented, was recalled, hence increasing the possibilities for CBI to 
become more active globally. 

With the Minister for Foreign Trade and Development Cooperation entering office in late 
2012, an ambitious policy agenda 54 was introduced that focused on coherency between trade 
and aid. It mentions, among others things, that despite diminished tariff barriers over the 
years, exporters from developing countries are facing stiffer EU entrance barriers as a result 
of non-tariff barriers, such as requirements on safety standards, energy-efficient production 
methods, durability and packaging. While Dutch PSD policies defend these more stringent 


50 See also 'Tweede Kamer, 2008-2009; aanhangsel van de Handelingen. Vragen nr. 3225 en 3586’. 

5 ' CBI proactively but unsuccessfully suggested to coordinate projects financed within the framework of 
the Schokland agreement (2009), and, more recently, proposed that it become the coordinating body of 
the technical assistance part of the Dutch Good Growth Fund. 

52 Internal memorandum DGIS, DDE-56rj/20io. 

53 Although CBI reports on additional jobs, this indicator has never become relevant for financial 
settlement. 

54 'Wat de wereld verdient: een nieuwe agenda voor hulp, handel en investeringen’, Tweede Kamer 20012-2013, 33 2 
625, no.i. Translated into English as: 'A World to Goin: A New Agenda for Aid, Trade and Investment'. 
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product admission criteria, on the one hand, they also aim to help exporters from less- 
developed regions to mitigate the problems they face because of this. CBI is specifically 
mentioned as having to render such support services in developing countries. 

A more proactive course taken by CBI led it to cooperate with international and national 
PSD organisations in a more structured manner. 55 Despite good intentions, as shown in the 
country studies, CBI is unfortunately seen as a ‘standalone’ service provider, not actively 
searching for or engaging in joint programmes that require more strategie approaches in 
country and sector promotion. 56 The reports also indicate that during the period under 
evaluation, the CBI programme was developed and implemented autonomously, i.e. 
independent of the respective governments’ national development strategy and MFA’s 
country programme (as implemented by the embassies). 57 

3.4.3 Rethinking CBI’s status 

During the period of evaluation 2005-2012, CBI was in full operational swing and took basic 
decisions about its future course and work methods. The transformation into an agency 
that is responsible for its own cost and expenses regime and that should operate under a 
result-oriented management regime since 1998 took longer than expected. 

The years 2005-2008 marked a period in which CBI management focused strongly on the 
reduction of a wide array of activities to four main billable products, while at the same time 
introducing a result-oriented management model. 58 A start was made to change CBI from an 
input-driven to an output-driven organisation. Five main priorities were formulated that 
CBI needed to act on: strategy and development, administration and Financial 
management, collaboration and coalitions, and monitoringand evaluation. 

After taking office in 2008, a new managing director decided to change course. It was 
recognised that too much focus had been laid on internal processes and product supply, 
without creating coherency between CBI's various departments. There was a strong need for 
clear organisation-wide goals to guide the activities of CBI and its divisions. Moreover, the 

55 CBI initiated the PSD round table, even as the joint website PSD/Trade2gether, to stress cooperation 
and synergy amongst network partners. It started cooperation with market access partner 
organisations PUM and IDH, later getting involved with the SNV inclusive business programme in 
Bolivia. In 2011 CBI started the Trade Related Instruments Connected (TRIC), an international platform 
of likeminded trade promotion organisations. In a first step, cooperation among members aims at 
market intelligence of which the objective is to prevent duplication of work and to share knowledge, 
and sharing information regarding potential SMEs and potential importers or collaboration in market 
entry by geographical division. 

56 Unlike its counterpart SIPPO, CBI was less successful in its effort to collaborate with several 
development partners and their respective programmes, who work in value chains (e.g. USAID, IFC, 

GTZ, DFIDand DANIDA). 

57 Since 20n, more integrated partnership programmes with local service providers (agreed upon in 
MoUs) have been launched, in which a task division between both partners has been agreed upon. 

CBI focuses on topics that local trade promotion organisations are unable to deliver. 

58 For example, employees were required to account for their time, more as a disciplinary measure than 
an attempt to create a better basis for justifying the product cost structure. The system was not kept in 
place for long though. 
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cliënt, i.e. the deputy DGIS, called for greater effectiveness and efficiency, introducing the 
need for greater external visibility. The organisation was urged to become more active in the 
poorest countries and fragile States, and to reduce its activities in upper-middle-income 
countries. Moreover, CBI’s management endorsed the fmdings of an organisational review 
by MFA/ISB in 2009 59 that began a process to develop clearer strategy, measurable objectives 
and more synergy. 60 

An offshoot of this renewed strategy is the integrated work method that was introduced in 
2011. The organisation’s new course aims to make CBI's impact and added value clearer, and 
to develop a strategie country focus that includes the least-developed and fragile States. 61 
A lack of clear internal goal setting by CBI, but also by the cliënt, made it necessary to 
implement a monitoring and evaluation framework, which resulted in the ‘Result 
Measurement & Evaluation Plan’ in 2012, and the development of a real-time impact 
evaluation method since 2013. 

The organisation review by MFA/ISB (2009) mentioned the agency’s relatively small size, 
implying how urgent it is to rethink the scale of its operations. Efforts by CBI to get fmancial 
contributions from other (predominantly international) donors proved disappointing over 
theyears. 62 This accelerated talks about bringing CBI under the umbrella of NL Agency, the 
executive agency for Dutch international business development (and the forerunner of RVO, 
the Netherlands Enterprise Agency). CBI initiated this by proposing its owner and cliënt to 
set up a business case. As of January 2015, CBI became part of RVO, meaning that CBI ceased 
to exist as an independent agency of MFA. 


59 Verkorte Organisatiedoorlichting Centrum tot Bevordering van ie Import uit Ontwikkelingslanden (CBI)’, MFA/ISB, 
2009. ISB is part of MFA. The acronym stands for'InspectieSignalering en Begeleiding’. 

60 See also the guiding document by CBI: ‘Koers 2010'. 

61 CBI annual plan 2010. 

During the evaluation period, CBI received an order from Ecuador (US$ 772,000) and a repeat of the 
order in 2015 (US$ 2.2 million) 
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3.5 Findings 

• The number of Export Promotion Organisations in developing countries has increased 
considerably in recent decades. They deliver similar services worldwide that should have 
triggered CBI to become more selective aboutwhat initiatives to support or not. 

• CBI has been sensitive to issues that negatively influence the programme's effect, and as 
a result CBI has reflected critically about which countries and economie sectors to select 
for its activities. A review of country and economie sector selection procedures in the 
ECP programme shows that not only ‘preferred’ countries were selected, but also 
countries marked as ‘doubtful’ or ‘not selected’, without making such choices explicit. 

• Since its interventions focus only on a couple of major export constraints, CBI has been 
unable to address other constraints that prevent companies from exporting. In other 
words, there was ex ante a considerable risk that the programme’s efforts, even if it was 
highly successful in bringing companies in contact with buyers, would not increase 
export turnover. 

• CBI’s transformation into a departmental agency in 1998 did not lead to the intended 
results-oriented management model supported by a cost-benefit administration. It has 
never been easy for any of the parties involved (the deputy Secretary-General being the 
owner, deputy DGIS being the dient and CBI the contractor) to adjust to their new roles. 
As result, MFA was unable to steer CBI towards efficiënt programme delivery. 

• MFA challenged CBI in 2010 to make the link between export promotion, sustainable 
development, and poverty reduction more tangible and transparent, after a period of 
sustained laissez-faire in which CBI could operate in splendid isolation. 


V* 




CBI’s contribution to export development 


4.1 Introduction 

This chapter focuses on the effectiveness of the export coaching programme. Section 4.2 
starts with a short introduction on how these effects are measured. Section 4.3 assesses 
both the extent to which ECPs have been able to relieve export constraints, as well as the 
effect on export turnover, also discussing the contribution of the programme in achieving 
these effects. Section 4.4 discusses how ECPs contributed to the achieved programme 
effects. Moreover, in order to have an impact on the national development indicators, the 
effects are assumed to be sustainable, and that the extra gains, in terms of added value and 
employment creation, will be, as much as possible, beneflcial for the national economy 
(Section 4.5). Reasons for not achieving the intended effect are given in Section 4.6. 


4.2 Measuring effects 

A review of the programme documents does not explicitly show how ECPs are supposed to 
function in particular contexts or how they will bring about the planned outcomes. Nor 
does it explain what causal mechanisms are supposed to be triggered or what the under- 
lying assumptions are for those causal mechanisms to operate, or what potential other 
pathways could explain the achieved results. Based on the reconstructed intervention logic, 63 
we have formulated our own assumptions (at the immediate, intermediate, and ultimate 
outcome levels, and at the impact level), which will be tested in order to have a better 
understanding of the programme’s effects. We will test these assumptions by looking into 
how the following indicators developed, and examining to what extent these developments 
can be attributed to the interventions received by the programme beneficiaries. 

At the immediate outcome level, ECPs should: 

• target companies most in need, which is reflected in the indicators 'programme 
additionality’ and 'company selection’; and 

• overcome key constraints faced by beneficiaries trying to achieve their export goals. 

At the intermediate outcome level, ECPs should: 

• contribute to (1) more organised and structured export planning and implementation, 
(2) an understanding of the European business culture, and (3) a more confident 
entrepreneur; 

• target both incumbent EU 6 ' 1 and new exporters, which is measured by the share of 
beneficiaries that started to export as a result of the programme; 65 and 

• diversify exports, both in terms of export destinations and product range. 

Annex 7 contains a visualisation of the reconstructed intervention logic. 

64 References to the ‘EU’ include ‘EFTA’ countries. 

65 CBI’s Memorandum to its principal dated July 3, 2008 proposed to replace the module name ‘export 
diversifïcation’ with ‘market development’. The new name reflects the tapping of new (i.e. European) 
markets for products that are already being exported, or placing new products on current EU markets. 

It no longer concerns the combination of new products on new European markets, because it is already 
known that this combination hardly leads to export growth. 
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At the ultimate outcome level the ECPs should: 

• increase the programme benefkiaries' exports and export share, measured by the share 
of beneficiaries that increased exports, and the size of these additional exports. 

At the impact level, ECPs are likely to: 

• support those sectors and companies that add substantial value to the domestic 
economy; 

• contribute to sustainable export increase and stable relationships with importers; 

• create additional (sustainable) jobs, while accounting for the potential short-run 
trade-off between competitiveness and jobs; and 

• prevent substitution effects, since from a macro-development perspective choosing a 
CBI-assisted company can be detrimental to current suppliers in developing countries. 


4.3 Findings 

When discussing effects it is important to remain focused on alternative pathways that 
could explain the results (sub-section 4.3.1) as well as any insights about the type of 
companies that entered the programme (sub-section 4.3.2). The programme effects at the 
intermediate, ultimate and impact levels are discussed in sub-sections 4.3.3 and 4.3.4. 

4.3.1 Additionality 

This section asks what inputs and services were provided by ECPs to companies in addition 
to what these companies could already obtain on the market or from other institutions 
operating in local markets, as these could offer alternative explanations for the results. 

The concept of additionality in its most basic form refers to the 'amount of output from a 
policy as compared with what would have occurred without government intervention’. 66 
Being additional means having added value or being complementary to what the market or 
other institutions are already providing (see Box 3). 


“ Mceldowney, J. (1997). Policy Evaluation and the Concepts of Deadweight and Additionality: 

A Commentary. In: Evaluation, April 1997, vol. 3, no. 2, pp. 175-188, European Commission, Brussels. 
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Box 3 Additionality defined 67 

The Independent Evaluation Group (IEG) of the World Bank investigated the 
additionality of the International Finance Corporation, a World Bank institution 
that provides investment and advisory services to build the private sector in 
developing countries. It States that additionality has three main characteristics: 

• Additionality is relative, i.e. it refers to what is available from other parties; 

• Additionality is situational, because availability depends on circumstances and 
‘situations exhibiting market and institutional imperfections that inhibit 
development’; and 

• Additionality is highly dynamic, i.e. the additionality of an institution may change 
as the institution itself changes, as other organisations in the environment 
change and as circumstances in developing countries change. 


According to our interpretation, we therefore need to ask whether or not ECP-assisted 
companies would have had access to a comparable and affordable alternative, in order to 
develop their export potential. This requires information on what type of export services 
were on offer during the intervention period, and to what extern companies had access to 
these services. 

The country case study reports reveal that many different types of export development 
services and subsidies were on offer by several institutions at the time of the intervention. 
Smaller companies, like those targeted by CBI, could generally not afford the consultancy 
services available in the private sector. Exporters that did not make use of other service 
providers stated that this was due to a lack of quality and availability of services, as well as a 
lack of ability to pay for these services. 

Moreover, investing in export development is often a low priority, and many companies 
even refuse subsidised trade fair participation because the required co-investments are 
deemed too large. This means that any export development services offered to these 
companies is additional to what they can (or are willing to) obtain on the private market. 
The question then remains whether ECPs were additional to the export development 
services offered by the local institutional framework. 

Our survey of 154 benefrciaries, drop-outs and non-benefrciaries indicates that almost half 
of the companies actually made use of export promotion services of organisations other 
than CBI since the ECPs started. 68 On average companies received 3.2 different types of 


67 IEG (2008) Independent Evaluation of IFC's Development Results 2008: lEC's Additionality in Supporting Private 
Sector Development. Washington, D.C.: Independent Evaluation Group World Bank. 

68 The fïnding that about half of the population has used export promotion service organisations is well in 
line with findings in the country case studies and other evaluation studies. 
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services since ECPs started. The most commonly received assistance is trade fair 
participation support, matchmaking, training on technical issues such as certification, 
custom procedures, training on personal skills, and one-on-one coaching and advice. On 
average companies paid EUR 1,700 for paid services, and in one third of the cases the 
services were freely available. A comparison of these other services with services offered in 
the ECP programme suggests that in almost 40% of the cases the companies perceived the 
ECP service to be totally different from the local services offered. Another 40% found the 
services 'somewhat different’, while 20% said the local services were ‘totally similar’ to CBI’s. 
Interestingly, when asked about the quality of the local services compared to CBI’s, 25% of 
the companies feit that the quality of the local service was lower than CBI’s, while the 
remaining companies either judged the local quality to be of the ‘same quality’ (38%) or 
even 'higher quality’ (38%). 

We conclude that half of the ECP beneficiaries also received services from other service 
providers. Most of these companies judged these services to be of equal or higher quality 
than that of ECPs. 

4.3.2 Selected companies and programme drop-outs 

In order to get a feel for the type of companies that entered the programme, the company’s 
total turnover has been listed (Figure 5), as well as the share of turnover that is earned out of 
exports, and the EU export share (Table 4). The mean average turnover is about EUR 23 
million, whereas the median is ‘only’ EUR 804,000, indicating a highly skewed distribution 
of ECP beneficiaries in terms of turnover. 

Figure 5 Company turnover at start of ECP intervention (logarithmic scale, n=/ 6 ) 



■ individual companies 


Source: IOB, survey. 


A closer look at the distribution in the above figure shows that this discrepancy is caused by 
a small number of outliers on the upper end of the distribution. The turnover for fourteen 
out of the reported seventy-six companies in the figure is above, and sometimes far above, 
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EUR i o million, with one Colombian company in particular reporting a turnover of EUR 1.2 
billion, of which EUR 240 million was export turnover. In other words, using the mean 69 for 
reporting purposes is not a good measure of central tendency. 70 

About 75% of the beneflciaries already had some experience exporting to Europe before 
entering the programme. The total export range of beneficiary companies, the median 
export share of total turnover, and the EU export amount are listed per quartile of 
observations (Table 4). 


Table 4 

Exports of beneflciaries the year before ECP started (n=76) 

Total exports 

Exports to EU 

Quartile 

Absolute (EUR 1 , 000 ) 

Median export share of turnover 

Absolute (EUR 1 , 000 ) 

1 

<30 

5 % 

0 

2 

30-239 

23 % 

1-18 

3 

239 - 1,824 

20 % 

18-182 

4 

1 , 824 + 

40 % 

182 + 


So urce: IOB, survey. 
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The data show that the export share of total turnover is positively associated with absolute 
export values, i.e. companies with a larger export volume also tend to have a larger export 
share of the total turnover. This supports the conclusion that larger companies are also 
more likely to reach the highest absolute export value, which is in turn the ECP indicator at 
the result level communicated by CBI to MFA. 

Another aspect is whether ECPs were offering the right kind of assistance for these 
companies to start exporting to Europe. Both survey and country case studies indicate that a 
lack of business contacts and a lack of market information were considered the most 
important reasons for not exporting or exporting more to Europe at the start of ECPs. 

Nevertheless, a large number of export obstacles that are less widely shared among 
companies remain, including, among other things, a general lack of organisational 
capacity, low European demand, EU import restrictions, unfavourable exchange rates, and 
not the right product in terms of quality or price. This indicates that there are many other 
reasons why companies were not exporting to the EU. 71 

69 Mean figures are always reported for ECPs. 

7 ° lf we lookfurther into the distribution of turnover, as graphically illustrated, we can see that at the start 
of ECP the majority of the companies (75%) had revenue that lay between EUR 100,000 and EURio 
million, while 13% reported an export turnover above EURio million, and 12% below EUR 100,000. 

71 Although not further spedfïed in the survey, the country case studies have given examples of 'other' 
reasons for not exporting more to Europe. An often-heard argument is a country's bad reputation 
(especially Colombia, India and Egypt) compared to other supplying nations. Companies appreciate the 
role CBI played in overcoming their reputation abroad, arranging their attendance at trade fairs in CBI 
pavillions, for example, instead of the home country, enabling them to avoid negative connotations. 

A decline in EU demand as a result of the economie crisis is not an argument that holds true forall 
Products and services addressed by the ECPs (see, e.g. the South African country report in Annex 4). 
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Table 5 Main reason for limited exports to the EU (multiple response, N=131) 

Reason 

Frequency 

Lack of business contacts 

47 

Lack of market information 

36 

Other 

100 

Total 

183 


So urce: IOB, survey. 

Beneficiaries may drop out of the programme intervention at various levels of the results 
chain, implying that the number that actually benefits will be smaller than the number that 
might potentially benefit. Of the 479 companies that were admitted to the thirteen ECPs, 

69 beneficiaries (14%) left the programme prematurely. 



CBI has not recorded the reasons why companies drop out of the programme, which makes 
it difficult to systematically analyse their motivations for doing so. Apparently the reasons 
are related to the selection criteria that are often used: capabilities (underqualified or 
overqualified), resources, and motivation of the company management. The case study 
examples suggest that companies were distracted by better opportunities in the domestic 
market, which eroded their commitment to export, while others seemed to be hindered by 
a lack of resources (people and money). 

4.3.3 Effectiveness in relieving export constraints (intermediate outcome) 

The simplified theory of change assumes that initially any company could become a 
flourishing exporter, were it not that certain constraints might prohibit this from 
happening. It further assumes that if these constraints are (to some extent) reduced by ECPs 
or by other external or internal factors, the company has a higher probability of increasing 
its export turnover. 

This section evaluates whether most of the mentioned export constraints were taken away 
during ECPs (Table 6) and to what extent ECPs contributed to this result. 
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Table 6 What was the most important problem at the start of the intervention solved at 

the end of the intervention? (I\l=99) 

Problem 

The problem 
is entirely 
solved 

The problem 
is partly 
solved 

The problem 
is not solved 

Total 

Lack of business contacts 

3 

18 

5 

26 

Lack of market information 

6 

15 

0 

21 

Other 

13 

20 

19 

52 

Total 

22 

53 

24 

99 


So urce: IOB, survey. 


We conclude that in about a fifth of the cases, the most important export constraints faced by 
participants were entirely removed during the intervention, while in half of the cases the 
problems were partly solved. For a quarter of the ECP participants, the main export constraint 
remained. It is remarkable that in about a third of the cases the export problems did not fit 
into any category, as these in particular have not been solved since the intervention started. 

An overall list of problems, and ECPs’ contribution in solving these problems, 72 indicates 
that ECPs were particularly successful in overcoming the lack of market information. About 
a third of the companies report that their main problem was entirely solved by ECPs, while 
for half of them it was partly solved as a result of ECPs and partly for other reasons. This is 
also confirmed by the country case studies and regular CBI evaluations, where a large 
majority of the participants reported that they acquired more skills and knowledge on 
export marketing and development, a better general orientation of European markets, 
better knowledge of characteristics and trends in European markets, and were better 
prepared for market entry. 73 The examples show that companies made improvements, 
especially in terms of preparing the company and product presentation for trade fair 
participation. 

‘Market access' refers to problems encountered with tariff and non-tariff barriers, such as 
certification. This is mentioned as a barrier for a minor percentage of the companies 
interviewed. Remarkably, since promoting ‘certification’ is considered to be an important 
aspect of the intervention, 74 we conclude that ECPs have been instrumental in general but 
hardly helped ECP beneficiaries to obtain certification for their products or services. 75 


72 The 75 companies who reported that their main export problems were entirely or partly solved were 
also asked whether this was because of ECPs or for other reasons. 

73 EIM, 2012. 

74 A separate ‘certification module’ has been launched as part of the new integrated approach, and was 
not offered as a separate module under the ‘old style’ programming. 

75 The contribution stories in the country reports often indicate that although it was the CBI expert that 
drove the decision to obtain certification, the companies themselves already applied for quotations 
before entering ECPs. In only one case is it clear that CBI shared the cost of obtaining certification. 




Aided trade 


Propensity to start exporting 

Since, according to the theory of change, ECPs are directed both at incumbent exporters and 
companies that had no earlier export experience, we are paying special interest to first-time 
exporters. 

Literature is not conclusive on the question whether promoting export helps firms to 
become exporters. 76 The evaluation of the Swiss Import Promotion Organisation (SIPPO) 
revealed that an estimated 5 out of 38 companies that started to export (of 76 targeted 
companies) could be referred to as first-time exporters because of the programme. 
According to the survey sample, the propensity to start exporting was 22% among 
beneficiaries, while a bit lower for non-beneficiaries (18%). This is caused by the share of 
companies that left the programme (drop-outs), 15% ofwhich did not export previously. 
These figures suggest that both beneficiaries and non-beneficiaries do not differ 
significantly regarding the propensity to start exporting. 

Effect on export diversifïcation (destinations and product range) 

If these export problems were solved, it would result in more customers and orders. Since 
the start of ECPs, 88% of the participants have found new importers that actually made an 
order. In the reference group, this was the case for two-third of the companies. According to 
a quarterofthe participants this achievementwas mainly the result of ECPs, while a third 
reported that this was partly the result of ECPs, but also partly for other reasons. A third of 
the participants reported that the new importers were found mainly as a result of other 
reasons. On average less than half of the new importers are based in the EU. 

About 84% of ECP participants reported that they have exported to new countries since the 
start of ECPs. This was 73% among the reference group. A quarter of ECP companies 
reported that they entered these new countries mainly because of ECPs, while 40% reported 
that ECPs partly contributed. Including those companies that already had exported 
somewhat to the EU, regular programme evaluations by CBI indicate that about half of the 
companies surveyed had at least one new export market (country) in the EU. 77 

According to the intervention logic, developing countries’ exports tend to concentrate on a 
few products, often commodities with fluctuating demand. This translates into high 
income instability, which in turn provokes high income volatility. The diversifïcation of 
markets and product supply has the advantage of creating a more stable income inflow. 
Market development can also contribute to general export growth, and flnally there are 
spillovers that benefit the larger economy as a result of having a more diversified 
production structure. About 75% of ECP participants diversified their product range after 
starting the ECP programme, which is a similar percentage to the reference group. 20% of 
the ECP companies reported that they diversified their product range mainly because of 

76 While Görg et al. (2008) and Bernard and Jensen (2004) find that there is no effect on the propensity to 
start exporting in, respectively, Ireland and the United States, Cruz (2014) fïnds that in the case of Brazil 
non-exporters are 130% more likely to export after receiving support. Schminke and Biesebroeck find a 
more modest effect in Belgium, where there is a 10% higher propensity to export. 

77 EIM, 2012, 2011. 
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ECPs, while 43% reported that ECPs partly contributed to diversification, and 33% reported 
that other reasons were mainly responsible. 

Based on the above observations, we conclude that ECPs made a marginal to modest 
contribution to export diversification. 

4.3.4 EfFectiveness on export turnover (ultimate outcome) 

In order to become a qualified 'competent exporter', CBI-assisted companies need to meet 
quality indicators at the result, effect, and impact levels. At the result level, the company 
indicates that its skills and knowledge have improved, and that it used its knowledge and 
insights of the EU market. The outcomes at the result level, as defmed by CBI, will not be 
further discussed here. 78 

This section discusses the effect ECPs had on export turnover. At the effect and impact levels, 
CBI quality indicators verify that the company increased exports to the EU during the 
programme's duration, of which the total additional export turnover to the EU is referred to 
as the achieved impact. 

According to CBI’s project administration, 410 beneficiaries out of the the evaluated ECPs 
were reported as ‘competent’ exporters. However, CBI’s project administration does not 
contain fmancial data for a substantial number of competent exporters (no EU export figure 
is reported for 17% of the beneficiaries, while the percentage for non-EU exports is 20%). 79 
Moreover, in light of the criteria for success at the effect and impact levels described above, 
CBI seems to be applying these criteria inconsistently. Finally, the programme management 
data does not support the evidence for declaring these beneficiaries ‘competent’ exporters, 
nor the claim for fmancial settlement as a result. 80 

Apart from the 20% of companies with no declared exports, which are discussed above, CBI 
project administration indicates that for another 20% of the cases, beneficiaries did not 
achieve additional 81 exports at all. 8z At the other extreme, about 10% (predominantly larger) 
companies achieved additional exports to both EU and non-EU countries of more than EUR 
1 million. The literature questions the additionality of the programme for large companies, 
since these companies are supposed to be self-sustainable due to lower unit market entry 
costs, for example, and larger sales volumes. When leaving out both extremes in the first 
and last rows of the table below (EU exports < o and > EUR 1 million) there remains a group 


78 Reference is made to the country reports. 

79 ECPs with the highest share of undedared exports are (1043) 'Pipes and process equipment’ (30% 
undeclared) and 'Office and school supplies’ (32% undedared). 

8 ° Delivering an ‘X’ number of competent exporters is the basis of the contractual agreement between CBI 
and MFA. The number of delivered companies forms the basis for fïnancial settlement. 

8 ' The word ‘additional’ is emphasised, as our review indicates that in a large number of cases CBI experts 
have included fïgures on total exports (instead of additional, defïned as in addition to a baseline export 
turnover figure for the company). 

82 The percentage is likely to be an underestimation, as the figure only reflects the companies for which 
fïnancial data are available. 
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of 177 (50%) companies with declared export increases. 83 Interestingly, compared to 
middle-income countries, a large proportion of the companies from upper-middle-income 
countries failed to achieve any exports, or saw a decrease in exports. Moreover, the data 
suggest that a higher number of companies from middle-income countries did not achieve 
exports at all (not in table). 


Table 7 EU and non-EU exports according to CBI database (N=339) 


Non-EU/EFTA export 

Total 

EU/EFTA export 

< EURO 

EUR 1 - 

EUR 500,000 

EUR 500,001 - 
EUR 1,000,000 

> EUR 

1,000,001 

N 

s EURO 

69 

22 

2 

11 

104 

EUR 1 - EUR 250,000 

19 

69 

8 

14 

110 

EUR 250,001 - EUR 500,000 

6 

12 

8 

11 

37 

EUR 500,001 - EUR 1 , 000,000 

5 

9 

3 

13 

30 

2 EUR 1 , 000,001 

23 

10 

2 

23 

58 

Total 

122 

122 

23 

72 

339 


Source: CBI project database (SAGE). 


According to CBI, the ‘additional’ exports for all 339 beneficiary companies amount to EUR 
290 million for EU exports and over EUR 700 million for non-EU exports, suggesting an 
internal rate of return (IRR) of over 40 times the invested ECP programme budget. Apart 
from the IRR definition used above, there is good reason not to base the IRR performance 
indicator on a comparison between programme expenditures and export turnover, and to 
use proflts or added value instead. 

Since the result indicator, as communicated with the cliënt, mentions the average export 
turnover achieved by all companies in a given ECP, it is likely that outliers may drastically 
disturb the underlying picture. 83 In light of the fact that the distribution of export turnover 
within (and across) ECPs is imbalanced, a more appropriate measure of central tendency 
would have generated a more realistic picture of the programme results. 

According to the country case studies, regular ECP evaluations, and surveys, which covered a 
longer time span than the usual one year following intervention, the increase in additional 
export turnover is modest, 85 whereas the distribution across participants is heavily 


83 When we express this as a percentage of all competent companies (including those for which no data 
have been included), the share drops to 43% (companies with a declared export increase). 

84 Table IV.8B, EIM 2011/12, provides a good illustration of the skewedness of export turnovers realised 
within ECPs. 

85 See for example APE (2014), which concluded that for seven out of twenty-three fïrms that exported to 
Europe, the export value in 2013 was smaller than EUR 20,000. EIM (2012) concluded that most 
respondents of the survey already showed a high level of exports to the EU in 2006, and that additional 
growth of export to the EU and EFTA has been moderate. 
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imbalanced, i.e. a small number of companies in an ECP show high export increases, hence 
increasing the general average ‘additional’ export. 

Of the forty-eight companies with complete Financial data in the enterprise survey, ten 
beneficiaries experienced a decrease in export turnover, whereas eight companies 
maintained a status quo. Of the remaining companies, six reported an increase of over EUR 
5 million, including two outliers that reported an increase of more than EUR 50 million in 
EU export turnover between the start of ECPs and the First quarter of 2013. The quite large 
concentration is notable, and resembles the Fmdings in other evaluations, including 
SIPPO’s, which found that ten out of thirty-three exporting companies achieved 70% of 
their overall turnover through exports, with four companies achieving a total of 40%. 

The surveys and case studies suggest that 44% of the competent exporters 86 increased their 
exports, and the effects ranged from small to large (Figure 7). 

Figure 7 Beneficiaries with export effects (all effect sizes) 


selection into the programme (100%) 


dropouts ( 14 %) / competent exporters (86%) 


I 65 


effect (all effect sizes, 44 %) 


2i out of 48 companies companies increased their exports. This equals 38% of those companies initially 
selected into the programme. 
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4.4 Programme contribution 

Having determined the outcomes in terms of change in export turnover, the question 
remains which part of the achieved effects can be attributed to ECPs. 

Box 4 Assessing contribution 

A contribution score of ‘4’ would typically refer to a case in which CBI helped the 
cliënt to understand the EU market, creating awareness of the position of its 
product vis-a-vis the global supply on offer. As a consequence, the company would 
reposition its product or service, and eventually increase its exports, selling to 
clients met during ECP-supported market entry activities. 

A contribution score of ‘3’ would typically refer to a case in which CBI helped the 
Client to understand EU market demands, creating awareness of the position of its 
product vis-a-vis its direct competitors. Nevertheless, the programme interventions 
could not remove major constraints faced by the company (e.g. certification) or 
constraints were removed by other factors, and the company would not benefit 
from ECP-supported market entry interventions (trade fairs, matchmaking). 

A contribution score of ‘2’ would typically be given to a case in which CBI 
accelerated a development path that was already initiated before entering the ECP 
programme, but where exports were increased despite ECP intervention. For 
instance, this is the situation for a company where an already experienced manager 
took the opportunity to develop or further develop an innovative product (on his or 
her own behalf), already having a good impression of customer needs and 
preferences, and already having a good understanding of global competition. 

A contribution score of ‘1’ would typically refer to a case in which CBI has opened 
the eyes of the company, but also made the company realise that it is not ready to 
export yet. Additional exports to non-EU areas were partially caused by increased 
market knowledge obtained at the trade fair and EXPRO, but entry into these 
non-EU markets were the company’s own efforts. 


Source: IOB. 
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A total of sixty-two companies (fifty-frve competent exporters and seven drop-outs) 87 were 
interviewed during the case studies in Colombia, Egypt, India, and South Africa. The 
contribution of ECPs was assessed for each of the companies according to the methodology 
described (Table 8). 8S Each company was given a contribution score in the country case 
studies, and a contribution narrative was developed for each of the beneficiaries with export 
effects to the EU and non-EU. 


Table 8 

Size effect and contribution of the effect for EU and non-EU (N=62) 

Size effect 

...of total 

...of which EU effect size due to 
programme = substantial 

...of which non-EU effect size due 
to programme = substantial 

Large 

10 

6 

5 

Moderate 

14 

2 

5 

None/weak 

31 

0 

0 

Drop-out 

7 

0 

0 

Total 

62 

8 

10 


So urce: IOB, country case studies. 


A moderate to large effect was realised by about 40% of the companies interviewed, which is 
comparable to the survey results. 50% of the companies interviewed either realised no effect 
or a weak effect, and over 10% of the companies interviewed dropped out of the 
programme. When combining these effect sizes with the assessed contribution by the 
programme, as described above, a small share of the beneficiaries had a substantial 
contribution in combination with a large effect size. 89 The intended effect with regard to EU 
exports for the beneficiaries interviewed hovers around 13-15%. 90 For exports to the non-EU 
area, the effect seems to be slightly higher. 

In the survey, ECP beneficiaries 91 were asked to what extent they attributed a growth in 
export turnover to ECPs (Table 9). We can see that 60% either did not achieve a growth in 
export turnover or could not give an answer. Of all beneficiaries, some 20% perceived the 
contribution by ECPs as considerable, or even higher. 


87 This is a representative sample at the 95% confidence level. 

88 For detailed contribution narratives and background information on the findings, reference is made to 
the country case studies. 

89 Large is defïned as > EUR 145,000 additional export turnover since the start of the intervention. 

90 15% when expressed in terms of total delivered companies, or competent exporters. 

9 ’ There is little overlap between beneficiaries interviewed during case studies and the ones that 
responded to the telephone survey. 
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Table 9 The extent to which participants attribute the growth in turnover to the ECP 

Extent 

Frequency 

Not at all ( 0 %) 

2 

Justa bit ( 1 - 30 %) 

14 

Considerably ( 31 - 60 %) 

17 

To a large extent ( 61 - 90 %) 

5 

Almost entirely (> 90 %) 

2 

Don't know/no increase 

69 

Total 

109 


So urce: IOB, survey. 


Asking a question such as the one above necessarily means that the answer will be based on 
the self-perception of the beneficiary. This perception can be biased as a result of the 
companies’ experiences or expectations of the results the programme should have 
produced, and whether or not these results met expectations. When controlling for the 
benefidary's export experience to the EU, a more nuanced picture emerges. Companies that 
express the highest contribution by the programme are the ones with no export experience 
to the EU, whereas those beneficiaries with experience are more reserved about the 
programme's contribution. This seems to be logical, as the more experienced companies 
might have higher expectations, as compared to their colleagues who never exported 
before. 25% of the companies without former EU export experience consider ECPs to have 
contributed to a large extern or even entirely, as compared to 11% of the companies that had 
previous export experience. 

Consequently, following the reasoning of establishing impact by CBI, the contributed 
additional export turnover is confronted with the expenses. The ECP quality indicator ‘total 
realised export to the EU markets' scores sufficiently if the total value of the achieved 
additional export is at least the same or more than the planned export. The Standard for the 
planned export is 2.5 times the planned CBI investment. The export turnover is measured 
from the start of CBI intervention until one year after completion of the programme. 92 The 
IRR, based on average expenses per company and total additional exports, delivers the 
following distribution of companies by number by ECP and is highly concentrated at a 
limited number of companies (Table 10). 


52 CBI theory of change and indicators for IOB evaluation, February 2014. 
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Table 10 Internal Rate of Return: count of beneficiary across ECPs (N=88) 


Export Coaching Programme 


Attributed IRR > 2.5 


No 


Yes 


Engineering products 

2 

2 

4 

Food ingredients 

5 

3 

8 

Home decoration accessories and gifts 

11 

1 

12 

ITO services 

6 

0 

6 

Medical devices and laboratory equipment 

6 

2 

8 

Mobile equipment parts 

3 

1 

4 

Office and school supplies 

8 

2 

10 

Macedonia outerwear 

6 

1 

7 

Personal protective equipment 

7 

0 

7 

Pharmaceutical products 

3 

2 

5 

Pipes and process equipment 

11 

2 

13 

Subcontracting casting and forgings 

4 

0 

4 

Total 

72 

16 

88 


Source: IOB survey, financiat data CBI. 
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According to the classification in the table, the intended effect, defined an IRR of 2.5 or 
higher, was achieved for 16 of the 88 beneficiaries (15%)." This confirms the fmdings from 
the country case studies that only a small number of companies benefited from the 
intervention. 94 


53 Corrected for companies that dropped out the programme, the conhdence interval hovers between 
10-20%, with a Standard error of 0.04. 

54 ECPs with the relatively highest number of companies with a positive IRR are in ‘Food ingredients’, and 
‘Engineering products and 'Pharmaceutical products’. Relatively more companies with a positive IRR 
can be also be found in the 'Medical devices and laboratory equipment’ and ‘Pipes and process 
equipment’ ECPs, though one has to realise that the products in this sector are more Capital intensive, 
which makes it easier to surpass the IRR threshold. 
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Figure 8 Beneficiaries with contributed effects at uhimate outcome level 


selection into the programme (100%) 


dropouts ( 14 %) / competent exporters (86%) 


effect (all effect sizes, 44 %) 


effect (> EUR 145 , 000 , 15 %) 


4.5 Developmental relevance 

Now that we know to what extent ECPs helped to increase the assisted companies’ export 
turnover, we need to try to assess the impact of the ECPs on sustainable economie 
development. For the saké of simplicity, we deflne sustainable development as a 
combination of economie growth and the creation of jobs. The gross domestic product 
(GDP) can be defined as the sum of consumption, investment, government expenditure and 
net exports. This implies that only the percentage of additional export turnover, the value of 
which is added to the national economy, increases GDP. The effects of the increase of 
exports on economie growth will increase more if there is little substitution and more value 
is added domestically. 

In addition to the effects on economie growth, the effects of ECPs on employment creation 
were assessed as well by conducting a contribution analysis as specified in the previous 
section. 

The above-mentioned illustrates that sustainability can be looked at from different 
perspectives, ranging from macro-socio-economic via economie sub-sectors (meso-level), 
to the individual company at the micro-level. Since ECP interventions are globally 
dispersed, with on average fifteen companies assisted per country in the framework of the 
thirteen ECPs under review, one could state that already from the outset there has been an 
inherent risk of the portfolio becoming too diffuse. Therefore, it would be going too far to 
claim that these interventions would have had a noticeable effect at the meso level, let 
alone at the macro-socio-economic level. 

In other words, given the setup of the programme and the means spent, it would have been 
unrealistic to expect the intervention to contribute to sustainable economie development 
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from the outset. 95 For this reason, in this section we prefer to focus on the company (micro) 
level, since this is what the interventions address. 

4.5.1 Sustainability of exports 

With a better understanding of the programme's effect, one must pose the following 
important question: to what extern did the benefits (in terms of exports) continue after 
CBI's interventions ended? 

When looking at outcomes at the intermediate level, regular CBI evaluations state that 
participants indicated that they gained more skills and knowledge with regard to export 
marketing and development, better general orientation on European markets, better 
knowledge of characteristics and trends in European markets, and were better prepared for 
market entry. 

At the ultimate outcome level, CBI does not keep track of the continuity of export relations 
that were generated by the programme. Therefore, no insights can be gained about the 
sustainability of the intervention from the programme administration. 

The regular programme evaluations on behalf of CBI are not able to look beyond one year 
following completion of the intervention. A sample comment in one of the ECP evaluation 
reports is that ‘at the time of the evaluation, there is not sufficiënt information to assess to 
what extern the achievements are sustainable. Many companies seem to have started to 
establish ongoing relations with EU companies, which is a good starting point for 
sustainable export relations. But the companies have notyet been able to enlarge the scale 
of their business with the EU. There is no information about the extern to which 
programme participants have shared their knowledge within their company'. 96 

Nor is there much evidence from the literature about the sustainability of the effects of 
export promotion. 97 A recent communication by ITC-WTO provides some specific insights 
into this topic in low-income countries - countries where CBI has been hardly active. 'SMEs 
with greater “internationalisation” tend to report higher turnover and growth and are more 
productive than their counterparts. But in low-income countries, 7 out of eveiy 10 new 
export relationships fail within two years. Only a minority of SMEs manage to sustain 
adequate growth patterns for a significant period of time. The SME sector is characterised by 
a high level of‘churn’ of exit and re-entry’. 9 ® 


55 Seealso the recent advice by the Dutch Advisory Council on International Affairs 'Ongelijke werelden, 
armoede, groei, ongelijkheid, en de rol van internationale samenwerking', which States that 'Some parts of the 
Dutch private sector/business development programme, which are now settled on a project basis for their 
impact on poverty, could be made clearer in their entirety on the poorest groups and regions.’ AIV, 2012. 

56 Ecorys, 2011. 

57 Cadott et al. (20i2)forms a notable exception, and findsfor an export matching grant in Tunisia that the 
increase in the export growth rate is only significant for one year following the intervention. Moreover, 
even export levels are no longer significantly different from similar companies that did not receive 
support three years following the support. 

98 ITC-WTO, 2014. 
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Figure 9 Effects on case study companies (N= 62) 
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Source: IOB, country case studies. 

We found that positive indications for sustainability increase export volumes and export 
shares of total sales over time. Negative indications for sustainability are irregular export 
turnovers, or even one-time exports, companies continuously facing cash-flow problems, 
and companies that do not seem to be fully committed to export, for instance because they 
72 | are too occupied serving the domestic market. 

The South African and Colombian case studies indicate temporary export increases for most 
of the assisted companies. Although one should be careful interpreting this, as the 
backdrop in export turnover by comparable non-assisted companies could have been even 
higher, it is clear that export turnover has been irregular. CBI reported that EU exports by 
Colombian companies should be qualified as temporary one-off sales during trade fairs, 
since none of the assisted companies continued exporting to the EU." 

A small number of Egyptian companies increased EU sales, while the intervention was only 
deemed to have made a contribution by one of these companies. For companies that feit 
CBI made less of a contribution (or none yet), the effect of participation tapered off rapidly 
following the programme. In India, more than half of the beneficiary companies were 
found to be on the path to sustainability. 

One way of looking at sustainability is to review whether relations between exporters and 
importers continued after the intervention ended. Although non-beneflciaries obviously 
did not get the intervention, they were also asked to provide their insight into the stability 
and continuity of their business contacts (Table 11). 


99 


Only one company did have a significant contract with a Swiss fïrm during the CBI programme (and was 
therefore considered a showcase for CBI in Colombia), but these exports were actually unrelated to the 
CBI programme (contacts and negotiations predate the ECP) and the contract has ended by now. 
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The figures indicate that a larger share of the non-beneficiaries were still in contact with all 
the new importers that were met during the period of intervention. About half of 
beneficiaries are still in business contact with some of the newly acquired importers during 
the intervention. 


Did you sell anything to new importers between the start and the end of the 
intervention? (N= 132) 


IMon-benefïciary 


Yes, to all 
Yes, to some 
No 

Total 

Source: IOB, survey. 


ECP participant 


12 

4 

4 

20 


Dropped out/rejected 


36 4 

50 3 

18 1 

104 8 


One of the regular CBI evaluations indicates that only a minority (4%) of the beneficiaries 
thinks that their current network is completely adequate and 64% thinks that their network 
is somewhat adequate. The remaining respondents (32%) find that their current network of 
importers is inadequate. 100 

4.5.2 Added value 

In order to have an impact on national development indicators, development should aim to 
reduce the share oflowvalue-added commodity exports. Accordingto the survey, on 
average exporters import one fifth of the inputs they use to produce their export goods. This 
means that the amount of value added domestically, which directly contributes to economie 
growth, is estimated at around 80%. Note, however, that this is an upper bound of the 
actual value added domestically, as goods higher in the value chain might have been 
imported by domestic suppliers of the inputs. 

This figure is in line with data obtained in South Africa, where it was found that for eleven 
supported companies, 75% or more of the value is added in the country. Low added-value 
scores were explained by the fact that the company outsourced the production to an EU 
manufacturer, while in the other cases the cost price was mainly determined by expensive 
semi-fmished products imported from abroad. The average scores confirm that added value 
in the service sector is high, compared to manufacturing, for example. It should be noted, 
however, that the added value strongly depends on the type of products: some products 
require a larger share of foreign input than others. 


173 


APE, 2014. 
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4.5.3 Substitution 

If a CBI dient gets an order that would otherwise be fulfilled by a company from the same 
country or by a company from another developing country, the net developmental effect is 
limited. CBI is aware of possible substitution effects. A feasibility study of a new intended 
ECP deals with this aspect by looking at the size of the programme, for example, and no new 
ECPs are organised in the same sector if the market appears to be tight. 

Sixteen importers that did business with ECP participants were asked with whom they did 
business before getting into contact with the CBI dient. Notwithstanding the fact that the 
number of observations was fairly small, the multiple response outcomes indicate that the 
goods or services now bought from ECP participants were previously bought from exporters 
in the same countries in a quarter of the cases, or from exporters in other developing 
countries in almost half of the cases. In such a situation, one may conclude that one 
developing country exporter is substituted for another, implying hardly any developmental 
impact at all. In almost half of the cases, it was also indicated that goods were previously 
bought from developed countries.' 01 

Another aspect of substitution refers to the geographical distribution of exports. An 
increased focus on exports to the EU can be detrimental for exports elsewhere, as has been 
shown in the case of the 'Pipes and process equipment’ ECP. A comparison of the 
development in exportvolumes to the EU and non-EU areas reveals that the size of the pie 
did not get any bigger for the companies enrolled in that programme, even if the European 
slice increased substantially. 102 Many other events may have influenced these results, such as 
market developments in and outside the European market. But it raises the question 
whether the increase in exports to Europe was achieved at the expense of exports to other 
parts of the world. 103 

Another aspect is that of institutional substitution, which is caused by a lack of donor 
coordination. As was indicated in the Colombian case study, there are other international 
organisations that promote exports from Colombia. The EU and USAID work on business 
competitiveness in general, and the EU and the Swiss development cooperation SECO have 
programmes to improve the business environment in Colombia, for example by 
strengthening the government’s systems for quality control or export registrations. 
Countries such as South Korea (KOTRA), Germany (German Chamber of Commerce) and 
Japan (JETRO) focus on the import of specific products intended for their countries, but with 
SIPPO, Switzerland has a programme similar to CBI. This has led to situations in which CBI 
and SIPPO interventions overlapped, though both organisations agreed later on to avoid 
such duplication. 104 

’ 01 As the question had multiple responses, the percentages add up to more than 100%. 

APE, 2014. 

’° 3 This phenomenon is not exclusively related to the ECP mentioned, but also comes back in the 
contribution stories of other beneficiaries, having not been active in this specific ECP and showing 
comparable trade-off effects. 

In the ECP ‘Macedonia Outerwear’, both SIPPO and CBI cooperated by organising a combined buyer 
mission and several training sessions together with USAID. 
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4.5.4 Employment 

One of the quality indicators that gauges the effect and impact of ECPs is the number of 
additional jobs created within the company measured one year after completion of the 
programme. The indicator is sufficiënt if the number of jobs has increased. 


Almost 90% of the beneficiaries reported some type of effect on employment, which is 
confirmed by survey results from regular CBI evaluations. However, when we control for 
ECP contribution in export increases, a different picture emerges. Of the companies that 
indicate that ECPs contributed to a considerable or large extent to increased exports, ECP 
led to increased employment in the form of full-time jobs in 15% of cases. Some reported 
increases in part-time jobs (9%), outsourced jobs (11%), and better working conditions (17%) 
(Table 12). 


Table 12 Effect of ECPs on employment (N= 318) 

I Effect 


Source: IOB, survey. 


Attributed to ECP (%) 


More full-time jobs 

53% 

15% 

More seasonal/part-time jobs 

31% 

9% 

More outsourced jobs in own country 

46% 

11% 

Better working conditions/other benelïts 

68% 

17% 

Other improvements 

70% 

12% 

No improvements 

9% 

0% 

Don't know 

4% 

0% 
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We briefly examined the effects on employment as reported by CBI and in the regular ECP 
evaluations, since these are communicated with the cliënt as a result indicator. 105 Our 
evaluation did not find evidence to underwrite CBI reporting, which suggests large job 
increases (Annex 11) instead. A closer look shows that the total of reported numbers is 
overestimated by more than 50%. 

Reasons for not achieving an effect on employment 

The survey and country case studies indicated that the dominant size class of assisted 
companies is micro or small. According to CBI/SAGE project administration data, the share 
of large companies that were assisted is apparently somewhat higher (Table 13). 


105 


The employment figures are also openly communicated, such as in the recent debate on the budget 
2015. 
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Company size-class distribution according to survey, case studies and CBI project 
administration 


Survey (N=l 26) 


Case studies (N=62) 


CBI/SAGE (l\l=479) 


Micro 

15% 

15% 

12% 

Small 

37% 

43% 

33% 

Medium 

41% 

30% 

41% 

Large 

7% 

11% 

14% 


Sources: IOB, CBI/SAGE. 


Since large firms tend to be engines of job creation, it is not surprising that only a small 
number of jobs were created. 106 According to the case study data, more than half of the 
medium-sized companies experienced an increase in employment development, whereas 
about half of the companies in the other size classes witnessed that kind of positive trend. 
Assisted companies with the highest contribution score are more likely to see their 
employment increase, indicating a positive association between contribution score and 
employment development. The South African case study showed that companies in the 
service and home decoration sectors employ a large share of the total staff on a temporary 
basis (e.g. freelance, home producers), creating a flexible base for their operations. 

Three micro-companies were assisted, which should be considered as inventing and design 
houses, that outsourced the complete manufacturing process. Supply chain spillovers, in 
terms of employment, are not known for these companies. What is known is that these 
employment effects (to a smaller or larger extern, depending on the company) take place 
outside the country. 

The Indian case study revealed that most companies employed fewer than ioo workers, both 
before and after the intervention. The reason why entrepreneurs do not officially hire more 
than ïoo employees seems to be the result of Indian labour laws that restrict retrenchment 
to this number of workers. This could dampen the increase in employment when the 
company might be considering to increase the size of the workforce to the extent that 
demand is perceived to be stable. 


106 This fact should not be taken to imply that CBI should concentrate on larger companies, since the 
contribution of the programme in especially these companies is modest. On the other hand, assisting 
more SMEs could also contribute to more job creation. 
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4 .6 Explanations 

In search for an explanation forwhy the intended effect was achieved only for a small share 
of beneficiaries, and not for the remaining 80% of the companies that entered the 
programme, we can look for answers at the macro, meso, and micro levels. 

Since potential reasons at the macro and meso levels have already been discussed (Section 
3.2), we would rather focus in this section on the micro level, since this is what ECP 
interventions address. We will look at both the explanations from the perspective of 
programme intake, selection, and implementation, on the one hand (Section 4.6.1), and 
company internal issues, on the otherhand (Section 4.6.2). 

4.6.1 Programme implementation and company selection 

Apparently, since the intended effect was achieved only for a limited number of companies, 
we must ask what can be said about the type of companies that benefited most. Since we do 
not want the answer to be biased by the self-perception of the beneficiary, we use the 
company audit assessments provided by CBI experts as a more neutral source of 
information. 

CBI does not use diagnostic tools for company selection apart from an initial audit. Clear 
procedures for identifying and selecting companies are absent, although officially a 
company has to comply with a range of selection criteria in order to qualify for 
participation. 107 This means selection depends strongly on the subjective opinion of CBI’s 
product consultants, who play a dominant role in the selection process. 

In order to join an ECP, a company needs to apply. Potential participants of ECPs are 
identified and invited to participate, in cooperation with CBI network partners in the 
respective country and sector. With the help of a CBI sector expert, the CBI programme 
manager pre-selects companies that meet the selection criteria and are eligible for 
participation based on the information provided on the application forms. 108 

Once a company is pre-selected, CBI sector experts will cariy out an export audit, evaluating 
factors deemed crucial to export success such as: production facilities and capacity; price; 
marketing and management skills; product features and packaging; certification; quality 
and safety; labour conditions; and compliance with consumer health regulations and 


’° 7 These selection criteria are: the company is at least 51 % locally owned, or by owners or co-owners who 
reside in another developing country (excluding developing countries characterised as UMIC or higher); 
it has 25 to 500 employees; it is no joint venture with a company based in a country with a classification 
of UMIC or higher; compliance or the willingness to comply with EU market requirements; no licensing 
commitments that prohibit or limit export possibilities of products to the EU; competitive prices and 
sufficiënt production capacity; a management which is able to communicate in English; the willingness 
and capacity to invest in adaptations of, for instance, product assortment and production processes, if 
and as required by the European market. 

108 CBI theory of change and indicators for IOB evaluation, February 2014 . 
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corporate social responsibility. 109 The export audit leads to an action plan, suggesting areas 
of improvement, required measures, and (business development) deadlines. 

Comparing the average audit scores for both beneficiary groups, i.e. with and without the 
intended effect for EU as well as non-EU exports, clearly shows that these scores differ in 
terms of statistical significance. Beneficiaries with the intended effect have particularly 
higher average audit scores in the following aspects: 

• management skills rating; 

• production process and ICT rating; company website rating; 

• sales history rating; profit rating; 

• export performance rating; export skills rating; export experience rating; 

• export packaging rating; and 

• order/quantity rating; delivery times rating; functionality rating; 

The above-mentioned aspects boil down to a company’s export experience, a conclusion 
which is also supported by the fact that the export share of total company turnover of 
companies with lower audit scores is also lower than that of companies with higher audit 
scores, but this difference is not statistically significant. An example with explanatory 
comments on ‘Export skills rating’ made by CBI experts can be found below (Box 5). 

Interestingly, the average audit scores suggest that differences in the following aspects are 
less of an issue when comparing beneficiaries with and without the intended programme 
effect: price; quality; product range; cost price calculation; effectiveness of communication; 
trade offer; organisational structure rating; fmancial situation rating; access to credit rating; 
quality control and procedures rating. 


’° s The project management information received trom CBI provides the audit scores per company, based 
on the clusters mentioned, that are divided into more than 50 aspects. The data file received from CBI 
shows that these audit scores have not been consistently administrated by the experts. Moreover, the 
scores show that these have not been harmonised among the individual experts, as indicated by 
consistent outliers on all aspects by these experts. 
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Box 5 Sample comments on exportskills ratings by CBI experts 


Companies with intended effect: 

• all master both German and English | 650 people employed in factory, 550 of 
which in production 

• no problems at all 

• mr. X speaks English very well and is well informed about the market 

• two people involved in marketing and export | English language 

• people involved in marketing and export | all of them speak English 

• good general manager, speaks very good English, UK education 

• they have an excellent dedicated team 

• well equipped to allow further growth of exports 

• 2t+ years in export, three people involved 

• sixteen, English spoken! 

• two people involved, full time 

• four people involved in marketing and export | all of them speak English 

• four people in export out of 10 people in sales and marketing | all in export speak 
English 

Companies without intended effect: 

• an experienced sales and marketing team for the local market is available and is 
capable to further develop an export strategy | start to gather information in the 
CBI surveys; see www.cbi.eu and continue to look for potential customers and 
competitors and develop a good understanding of European product 
assortments | continue to search! 

• mr. X is working on his English 

• search with the help of mr. X for sales people, initially for domestic and regional 
markets 

• little English or other languages 

• the relatively large turnover of export staff might indicate that too much time is 
being spent on internal administration and too little time on market research 

• mrs X. speaks English and knows Italian due to colleague with whom she still 
works on ideas and on fabric buying 
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What this tells us is that companies with the highest audit scores seem to benefit most from 
the programme. 110 But at the same time, it also raises the question about the target group 
for which ECPs can have the most added value. In other words, the additionality of the 
programme could be at stake here. As was concluded in the literature review, the group of 
companies for which export promotion seems to be most effective are generally those with 
little export experience (Martincus and Carballo, 2010b; Schminke and Van Biesebroeck, 
2013). Intuitively this seems logical, as these companies have the most to learn about 
exporting. Moreover, the companies that seem to benefit most are mid-sized companies 
(Martincus et al., 2012) that have sufficiënt capacity to fully use governmental support, but 
just not enough to overcome market entry barriers on their own. 

From the perspective of programme implementation, we must then ask what kind of 
treatment both of the company groups received. 

The ECP intervention is subdivided into a number of sub-treatments (or modules): audit;" 1 
capacity building (technical assistance missions; 112 distance guidance" 3 and EXPRO” 4 ); 
market access (trade fair participation;" 5 buyer mission/showroom" 6 and consolidation" 7 ). 

According to the modules registered in CBI’s database (SAGE), companies with the intended 
effect, i.e. those with the highest audit scores, received significantly more market access 
(trade fair participation) and consolidation. On average companies with the intended effect 
received distance guidance 3 times, as compared to 1.8 for the other group. They also 
received trade fair participation on average 2.5 times and consolidation 2.1 times, as 
compared to 1.8 and 1.7 respectively for the companies without the intended effect. 


110 The reason why some of the companies benefited more than others could also partly be explained by 
reverse causality. This does not seem to be likely though, since we were able to control for audit scores 
at programme entry. Also, both company groups do not differ significantly on the distribution of 
achieved export turnover. 

One day assessment of a company on more than 50 different aspects. 

112 Although there are differences between experts and ECPs, the general picture is that companies are 
visited once (maybe twice) a year during the multinannual intervention. 

113 Intermittent guidance by the CBI expert. 

” 4 One-week training in the Netherlands for one company employee. This will familiarise them with the 
European market and equip them with practical skills on export marketing, export management and 
promotion in their sector in Europe. By the end of the seminar, they will have developed their own 
export marketing plan for the European market, for which they will receive (individual) counselling. 

115 Up to three trade fair participations organised and fïnanced by CBI. For the second and third fairs, 
participants are required to contribute to the costs for stand space and stand construction of 
respectively EUR 2,500 and EUR 3,500 (dependingon thefair’s nT price, for example). 

116 Only implemented in ECP 1035 ‘Macedonia Outerware’. 

117 Following B 2 B activities and trade fairs, the CBI expert will help consolidate, solidify and expand the 
business contacts. 
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It suggests that the group of companies to which ECPs contributed the least received the 
most intensive (and expensive) 118 treatment. Whereas companies with lower audit scores 
received less capacity building (TAM and distance guidance), which intuitively goes against 
the 'Working Instruction for Implementing ECP modules' (see Annex 6). 

Based on the distribution of audit scores, we subdivided the beneficiary companies into 
three new groups (average audit score per cluster" 9 in brackets): ‘overqualified’ (3.4), ‘target 
audience’ (2.58), and ‘underqualified’ (1.95). Or rephrased in other terminology,' 20 within 
the heterogeneous ECP population one could discern the following groups: (1) Tast milers’, 
who are self-supporting; (2) advanced companies already engaged in export activities, who 
ECP could assist by accelerating exports; and (3) companies with support needs stretching 
beyond the Tast mile’. 



Source: IOB, survey. 


Cross tabulation, combining the distribution scores with the assessment ofwhether the 
intended effect has been achieved, essentially shows that a relatively large share of 
companies qualified as ‘overqualified’ achieved the intended effect (40%), whereas this was 
only true of about one flfth of the companies defmed as the ‘target group’. Hence a large 
potential remains unexploited. 121 As expected, neither of the underqualified companies 
achieved the intended effect. The outcome underlines the finding that the investment of 
scarce resources was not optimally directed at the group of companies most in need of it. 

Crossing the data with company size class confirms the conclusion from the literature 
review that overqualified companies are often also the larger companies, whereas micro- 
sized companies tend to be overrepresented in the underqualified group. 


’ lS Price of export capacity development (module E) amounts to EUR 26,000; market access (module H) 
EUR 43,000. 

" 9 Scores rank from o (worst) to 4 (best). Only companies with an average score of 1 on critical aspects are 
eligible. 

Terminology based on COMO Consulting, 2011. 

Well-targeted benefïciaries are predominantly in the following ECPs: ‘Food ingredients' (1037), ‘Personal 
protective equipment’ (1038), and ‘Pipes and process equipment’ (1043). It seems that overqualified 
companies were selected in the ‘Pharmaceutical products' (1043) and ‘Medical devices and laboratory 
equipment’ (1030) ECPs. 
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When looking at the companies’ export experience, we find that 60 % of the overqualified 
companies had never exported to the EU before (though they did to non-EU destinations), 
whereas more than 70% of the underqualified companies had already exported to the EU. 
Indeed, company selection was flawed in two fundamental ways: 

• First, overqualified companies with no experience in the EU, but some experience in 
non-EU areas, are considered qualified and motivated to start exporting to the EU. 

• Second, companies that are essentially not ready to export (underqualified), but which 
proved capable of occasionally exporting to the EU in the past, are taken up into the 
programme on the assumption that they will be able to start exporting on a more 
structural basis. 

The fact that hardly any ECP beneficiaries are to be found in lower-income countries, 
reflects the perceived risk of ending up with an underqualified company. Target companies 
seem difficult to fmd in these countries. The majority of the defmed ‘target group' 
companies come predominantly from lower-middle-income countries, where additionality 
and attribution seem to be the highest. 

The outcome of the analysis reflects the deliberately diverse selection of weaker and 
stronger companies into ECPs (‘healthy mix’), in line with CBI’s assumption that the 
stronger companies will serve as a benchmark for the less strong ones. In addition, 
companies from countries with a weaker supplier profile or image could benefit from 
improved visibility and the reputation of stronger countries in the programme, for example 
during a collective exhibition. CBI’s assumptions underlying the ‘healthy mix' seem to be at 
least partly invalidated by the analysis. 122 

As some external experts remarked, the selection process could benefit from cooperating 
with local organisations, in order to broaden the base for company selection and thus 
increase the quality of the intake. Since this basis has obviously been too small in some 
cases, requirements were dropped, thus making it difficult to execute the programme as 
intended. 

4.6.2 Internal company reasons 

The following internal preconditions are necessary for a company to export: (1) willingness 
and motivation, and (2) being ready to export. ECP interventions focus on the latter: getting 
companies ready to export. 

According to the country case study results, half of the ECP participants achieved either a 
minor effect or none at all. There are several reasons why the intended effect was not 
achieved (Table 15). In about eight out of thirty-one of the cases the intended effect was not 
achieved due to a mismatch between the product and the intended export market, 


Since companies in ECPs will only meet sporadically during the intervention period (EXPRO and at trade 
fairs), and no such counselling between firms is organised as part of the intervention, it is also less likely 
for the expected effect to actually occur. 
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under-qualification, and not being ready to export. In another seven cases the motivation 
seemed to be a hindering factor, while other companies became distracted because of 
increased sales opportunities in the domestic or other markets. 

Since motivation is a serious factor, the selection procedure should attempt to understand 
these soft skills better. In another fifth of the cases, companies were overqualified, such as 
in Egypt, for example, where companies that were already competent exporters before the 
programme started were included in the programmes. In such cases, the companies were 
often disillusioned when the programme failed to live up to their expectations in terms 
services, which were not delivered, or because the company was still too far from being 
ready to export to Europe (e.g. because the company’s product did not meet the 
requirements of the European market). In these cases, the ECP was of limited added value. 


Table 15 Reasons for not achieving the intended effect (P 

J= 31) 

Reason 

Beneficiaries (case studies) 

Mismatch between offer and demand (product not suitable/ 
not competitive/company not export ready or underqualified) 

8 

Motivation/domestic orientation/distracted 

7 

Overqualified 

6 

Lack of finance/production capacity 

3 

Intervention not relevant 

2 

ECP premature cancelled 

2 

Unknown 

3 

Total 

31 


So urce: IOB, country case studies. 


An examination of the initial audit scores of the case study companies reveals the following 
picture (Table 16). The distribution scores confirm the idea that CBI has been focusing on 
picking winners. More than half of the companies with an audit score higher than three 
could be labelled as 'overqualified'.' 23 


Table 16 Company selection in case studies (N= 62) 

Audit score at intake 

Beneficiaries 

1-2 

2 

2-3 

12 

3-4 

18 

Unknown 

30 

Total 

62 


Source: IOB, country case studies. 


” 3 Out of the group with an audit score > 3, 40% had a score > 3.5, i.e. 21% of the total intake. 
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4.7 Findings 

• ECPs have been particularly successful in overcoming the lack of market information. 
The large proportion of other problems that remained unsolved, thus preventing 
companies from exporting, is striking. 

• Almost half of the companies actually used support services from organisations other 
than CBI, and half of those evaluated the quality of these local services as equal to or 
even higher than CBI’s services. 

• The sample data suggest that beneficiaries and non-beneficiaries do not differ 
significantly when it comes to diversification of exports (products and markets) and 
employment creation. 

• Increases in export turnover are modest for the majority of beneficiaries. The dominant 
share of additional exports is achieved by a small number of larger companies. 

• About 44% of the competent exporters for which export figures are declared achieved an 
increase in export. About 15% of the competent exporters increased exports 
substantially (> EUR 145,000). 

• CBI selects a ‘healthy mix’ of both stronger and weaker companies for the programme, 
assuming that weaker companies will benefit from the mix. The analysis does not 
support this assumption. Companies that benefited the least from ECPs received the 
most intensive (and expensive) treatment, while other companies received fewer of the 
needed interventions. 

• The majority of companies that benefited from CBI’s support come predominantly from 
lower-middle-income countries. 

• CBI's programme administration and programme evaluations do not give insight in the 
sustainability of results. The results, if any, are often temporary. Nor is there much 
evidence from the literature for the sustainability of the effects of promoting export. 

• The proclaimed employment effects as communicated to MFA are severely overstated. 

• There are indications of substitution effects, in terms of replacing a developing country 
exporter for another, implying limited developmental impact at the meso/macro level. 
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5.1 Introduction 

This chapter discusses the efficiency of export coaching programmes, both from an 
organisational perspective (Section 5.2) and at the programme level (Section 5.3). 
The results of an external (organisational) benchmark are reported in Section 5.4. 


5.2 Organisational efficiency 

Efficiency is a measurable concept, determined by the ratio of output to input. In order to 
address the efficiency question, it is necessary to understand the development of the 
programme budget and the cost drivers in the evaluation period (sub-section 5.2.1). 

When CBI turned into an agency, it introduced a cost and expense organisational model 
that presumes the handling of a full cost accounting model. An understanding of the 
development of cost prices resulting from the cost accounting model is instrumental in 
order to gain insight into the efficiency of the organisation (sub-section 5.2.2). 

5.2.1 Budget and cost drivers 

The development of the main cost drivers, i.e. material and personnel, had to be in line 
with the development of the programme budget received from MFA. 

Figure 10 Development cost drivers and programme budget (2005-2009) 



^-programme budget (excl. ITC) ^—personnel costs 

- material costs trendline (programme budget excl. ITC) 

trendline (material costs) 


The total costs of personnel increased until 2009, as can be seen from Figure 10, which 
describes the development of the development costs and programme budget between 2005 
and 2009. With the introduction of the new cost accounting model, all direct personnel 
costs were attributed to the specific products/modules, and became part of the ‘material 
costs'. 124 (After 2009, personnel costs only included personnel costs that cannot be 
attributed to a specific product (module), i.e. overhead.) Two trendlines are included in 

The term 'material costs’ includes more than costs of housing, office supplies and the like. It includes all 
costs that can be directly attributed to a product module. 
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the flgure, representing the ra te of increase in the programme budget and material costs. 

A comparison of the slopes of both trendlines makes it dear that the cost of running the 
programmes have been increasing at a faster pace than the programme budget itself. 125 This 
is an indication that the organisation did not become more efficiënt over time. 

5.2.2 Full cost model 

Effectiveness and measurability within a cost and expense organisational model requires 
the handling of a full cost model, in which the costs ideally should be based on a good 
understanding of the contribution of each of the cost drivers in the production (see 
Figure 11). The model is based on the presence, among other things, of homogeneous 
measurable products and/or services and presumes efficiënt monitoring using indicators. 
Insight into the relative efficiency of the implementation can be gained by comparing 
calculated production costs with costs from previous years or the cost of a product or service 
from other organisations. For this, it is necessary to relate the relevant costs of a service to 
its performance. Moreover, in order to improve efficiency, it is necessary to have a multi- 
annual insight into performance indicators in terms of quality and cost. Our analysis 
indicates that at CBI such an overview is still problematic. 

The cost accounting model needs to be rethought for a number of reasons: 

a) to line it up with changes that had been introduced in the operational country list 
(more focus on partner countries, LDCs and fragile States) 126 ; 

b) to meet the wish of the cliënt to introducé a more active steering of CBI; 

c) to address remarks that had been made by the ADR, which repeatedly requested the 
accuracy of the costs charged by CBI to be examined; and 

d) to address the need to introducé more flexibility into programme implementation. 127 

Until 2006, itwas common practice for CBI to clear fmances at the time of delivery against 
an ex post price for its products (ECP, BSOD, MI and HRD). Briefly, this meant that all costs 
incurred were charged to the cliënt. 128 Additional requests from the Ministry of Finance led 
to a change in clearance settlement (ex ante rather than ex post). 


115 Also, when the costs for personnel and material are added, it does not affect the difference in the slope 
ofthetwo trendlines. 

” 6 In the CBI country-sector statements for the years 2010 and 2011, however, twelve countries are 
reported that should have been excluded according to the MFA operational country list. These are: 
Bosnia and Herzegovina, Colombia, Costa Rica, Ecuador, Jordan, Macedonia, Montenegro, Peru, South 
Africa, Suriname, Thailand and Tunisia. It is unclear to what extent the operations in these countries fall 
underthe agreement that allows for a 15% deviation of the budget to be spent in non-MFA listed 
countries. 

” 7 For example, in the former situation CBI could not invoice the interventions delivered to companies 
that dropped out of the programme, losing the investment so to speak. 

” 8 Programmes executed in this period are so-called 'task T programmes. 
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From 2006 onwards, CBI and MFA agreed that in the new multi-annual programmes ECP 
and BSOD products would be settled at the time of delivery against an ex ante price. 129 The 
settlement price was based on allocated expenses using hourly rates for the programmes, to 
which a ‘risk’ 130 and ‘profit’ mark-up were added. This led to cost prices that were 
structurally higher than the actual programme expenses. 

In 2007 MFA approved a request from CBI to increase the budget per delivered competent 
ECP exporter by 17% (from EUR 56,750 to EUR 66,445). CBI argued that relatively cheaper 
ECPs had been cancelled, since they did not live up to the requested level of quality, the 
assumed savings in programme delivery costs did not materialise, and/or market 
developments requested more extensive coaching of beneficiaries by CBI consultants. Based 
on the documents reviewed, this was accepted by the cliënt without any further request for 
clarification. Another example of budget maximisation is shown in Box 6. 

Based on detailed evaluation of the intervention activities deployed at the company level for 
the ECPs that started since 1999, IOB concludes that the claimed intensity of ECP 
interventions was never really there, and that therefore the justification for increased 
financing per competent exporter was not empirically supported. 

Box 6 Budget maximisation' 3 ' 

For the purpose of better estimating CBI’s liquidity needs to implement both 
contracts (‘old’ ex post and ‘new’ ex ante regime products), the planned production 
was multiplied by the price (P) per unit (Q). As the P and Q were not quantifïed for 
the old regime products, it was decided to curb CBI’s production by allowing a 
maximum increase in either P or Q of 25%. In the accountability statement of the 
2008 production, which was approved by the ADR, the approved budget per 
competent exporter is set at EUR 57,543, which is 27% higher than the agreed price, 
and therefore not in line with the maximum variation set at 25%. This was accepted 
without intervention from the clients’ side. 


Up to 2010, CBI programmes were linked to a single product. This ensured that no 
programme that entailed a more comprehensive approach, for example by combining 
sub-products into a tailor-made product offer, could be launched. To ensure consistency 
with the introduction of the new programmatic work approach since 2011, the cost model 
was adjusted to allow for a more flexible form of programme delivery. Existing products 
were subdivided into modules, and the then current programmes were incorporated into 
the new modular approach. 


” 9 These are so-called ‘task 2’ programmes. 

130 Economie, political and cultural risks and natural disasters. 

131 Source: Memo 09/06/2006. 
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In order to achieve a more accurate cost price' 32 of programme modules, MFA and CBI 
decided that starting in 2010 price setting would be based on out-of-pocket expenses, to 
which were added a 10.5% mark-up on the net programme management costs and a 20.5% 
mark-up on the net programme overhead costs. Our analysis indicates that the mark-up for 
overhead is much higher than the actual costs of production, at least for two of the three 
years projected in the table (Table 17). 



So urce: IOB, based on CBI annual financial accounts. 

CBI saw its turnover and profit rate increase from 2009 onwards, with the profit ratio 
increasing from an annual average of a 0.5 percentage point between 2005 and 2008 to a 3.1 
percentage point between 2009 and 2012, which is more rapid than the increase in turnover 
of 36% over the same periods. The explanation lies with CBI’s introduction of the ex ante cost 
price settlement, combined with the transfer pricing model allowing for more flexibility, as 
well as mark-up cost prices, while CBI has notyet become more active in LDCs and fragile 
States. 

Figure 11 Development of net result (2006-2012) 



net result (excl. rebate) -trendline (net result excl. rebate) 


132 Actually, the term 'transfer pricing’ better reflects reality since it is based on real expenses, including 
mark-ups to cover overhead, risk, and profit. The term 'cost price’ suggests real expenses per unit, 
excluding these mark-ups. 

’ 33 Total of indirect costs of personnel plus material costs (excluding programme costs). Any personnel 
costs that can be attributed to a product have not been included. 
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A breakdown of these profits by product modules shows the following picture (Figure 12). 134 
The implementation of ECPs, especially the aforementioned 'Company audit and action 
plan’, 'Export capacity development’, and 'HRD training’ modules have been highly 
profitable. Losses on the following products could be easily offset by gains: 'BSOD export 
development promotion’ and ‘Market and tailored intelligence’. 

Figure 12 Product modules contribution to net result (2010-2012) 



Given MFA’s wish for CBI to become more active in LDCs and fragile States, one of the 
arguments underpinning the need to introducé cost price mark-ups was that operational 
costs to implement programmes were supposed to be higher in these countries compared 
to areas CBI used to operate in. Since CBI’s activities in LDCs and fragile States did not meet 
the client’s expectations, the mark-up cost prices were, in fact, too high to compensate for 
its activities, which took place predominantly in lower cost countries, and hence resulted in 
a rapid build-up of equity. 135 In order to prevent this from happening, the ‘Regulation 
Departmental Agencies' of the Ministry of Finance introduced a threshold above which 
excess Capital had to be transferred back to MFA, 136 while CBI proposed a discount of 20% on 
total ECP production in 2010,10% on 2011 production, and 5% for theyears 2012 and 2013. 137 


’ 34 Annex 8 provides detailed insight into profit/loss per intervention module. 

135 In 2013 the price level of the modules was reassessed. This reassessment has led to an adjustment of 
the prices of nine modules (Annual plan, 2014). This adjustment is approved by owner and cliënt, with 
the consent of the control unit. 

136 The regulation stated that CBI’s equity could not become more than 5% of its average turnover of the 
last three years. CBI has tried to doublé the percentage but, according to the documents reviewed, this 
was never granted by the Ministry of Finance. 

’ 37 In 2012 the cliënt was granted a discount of EUR 1.5 million to adjust profits downward, in order not to 
exceed the threshold. 
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Box 7 Is support in LDC countries and fragile States more expensive? 

One of the arguments behind the ‘cost price’ mark-up has been the fact that CBI 
should become more active in LDCs and fragile States, countries where production 
costs are assumed to be higher. A fïnancial analysis of ECP activities deployed in 
so-called ‘focus countries’ does not provide evidence for such a daim. In fact, the 
contrary has been proven to be the case: the costs per company in these countries 
are lower (EUR 56,000) compared to other country classifications (e.g. EUR 63,000 
in PSD countries). One of the reasons that might explain the average lower costs in 
‘focus countries’ is that the interventions primarily involve business development, 
and to a much lesser extent deploy modules in the heids of export capacity building 
and market entry. 
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Non-homogeneous measurable products and services 

As indicated earlier, effectiveness and measurability within a cost and expense model 
require the handling of a full cost model, which assumes the presence of homogeneous 
measurable products and services. Commercial work assumes control on the basis of 
products and services, and funding on the basis of production, based on full cost. 

Therefore, according to the scheme, it only makes sense to use an income and expenses 
model if clear products and/or services are identifled. In addition, the contractor’s activities 
need to be predictable, and the contractor must be able to influence the costs. 

Several evaluations, especially those of implemented BSOD modules, clearly indicate that a 
Standard module with a Standard transfer price do not go well with the demand-driven, 
context-determined needs of the beneficiary, which require tailor-made assistance. For 
example, BSOD modules had to be changed in India, while in Indonesia the key question 
was how the BSOD programme could complement and synergise best with other projects to 
get maximum added value for the BSOD. 138 


138 llumtics (2013) pp. 21-22. 
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The cost and expense model requires all actors to enter into a discussion on the price, 
quality, and quantity of the services to be provided. It makes more sense, in particular with 
organisations that provide heterogeneous and/or socially relevant products, to start from 
the available budget. 


5.3 Programme efficiency 

This section focuses on an efficiency analysis at the programme level. It provides 
Information about expenses, transfer prices, overhead, delivery rate, costs associated with 
drop-outs and rejected companies, employment change at the ECP level, and also compiles 
ratios based on the number of competent companies. 

Table 18 gives an overview of the number of competent companies per ECP, the amounts 
received per competent company by CBI, based on the pre-agreed prices with MFA per 
delivered programme intervention module, actual costs made by CBI per competent 
exporter, the overhead rate per ECP, costs related to companies that left the programme, 
and the average change in employment per competent company. 


Table 18 Programme costs and efficiency parameters 


Description 

Competent companies 

Transfer pricing per 
competent company 
(in EUR) 

Expenses per 
competent company 
(in EUR) 

Profït per competent 

company 

(in EUR) 

Overhead to 
expenses ratio 

Delivery rate 

Costs of drop-outs 
to total expenses rate 

Employment change 
per competent 
company 

Mobile equipment parts 

30 

51,251 

51,251 


21 % 

88 % 


22 

Engineering products 

32 

58,564 

58,564 


22 % 

82% 


20 

Medical devices and laboratory 
equipment 

37 

57,969 

57,969 


23% 

90% 


9 

ITO 

38 

41,938 

41,938 


22 % 

88 % 


15 

BPO 

10 

65,584 

65,584 


30% 

56% 


-10 

Pipes and process equipment 

47 

58,119 

58,119 


18% 

87% 


58 

Outerwear 

21 

73,381 

59,645 

13,736 

25% 

88 % 

1.3% 

-73 

Sub-contracting castings and 
forgings 

38 

79,126 

57,965 

21,161 

20 % 

88 % 

4.6% 

16 

Food ingredients 

29 

84,600 

64,511 

20,089 

24% 

76% 

5.3% 

0 

Personal and protective equipment 

21 

80,467 

70,557 

9,910 

25% 

76% 

4.4% 

n.a. 

Office and school supplies 

44 

86,455 

65,833 

20,621 

24% 

90% 

1.3% 

-4 

Home decoration accessories and gifts 

44 

79,205 

60,772 

18,433 

26% 

94% 

0.9% 

0 

Pharmaceutical products 

19 

86,200 

58,786 

27,414 

23% 

96% 

0.7% 

5 

Total 

410 

68,672 

58,393 


23% 

86% 

2.7% 

9 


Sources: CBI, IOB. 
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The table is split into two compartments. The ECPs above the marked line represent the 
ones that were financially settled ex post (task t assignment), and the ones below the line 
represent ECPs that were financially settled ex ante (task 2 assignment). The table illustrates 
that CBI started 'to make money’ after the ex ante pre-agreed pricingwas introduced in 2006, 
while overhead did not decrease. 

The first column mentions the name of the ECP, followed by the number of companies 
within that ECP. The 'transfer pricing’ column is different from the column on actual 
programme expenses, since an ex ante product pricing method was introduced halfway 
through the period of evaluation. Until 2006 the agreement was that only costs incurred 
were settled (task 1 assignment of CBI by MFA). Under the task 2 programmes, individual 
modules delivered per company were settled in advance with the cliënt against a pre-agreed 
price. This marks the point at which CBI actually started to make a profit on delivered 
programmes, up to EUR 4 million, 139 after which itwas decided to introducé a cap on the 
build-up of private equity. 

The 'profit delivered’ column shows the profit (the sum of transfer prices minus expenses) 
per delivered company. The 'Pharmaceutical products’, ‘Sub-contracting castings and 
forgings’, 'Food ingredients’, and ‘Office and school supplies’ ECPs have been highly 
profitable. The total average profit per company amounts to EUR 19,085, or 15% of the 
average total transfer price per delivered exporter. 140 

When looking at how expenses per ECP developed vis-a-vis the transfer price between ‘task 
1’ and ‘task 2’ assignments, it becomes clear that expenses raised by 13% from EUR 55,200 to 
EUR 62,500. 141 At the same time the average transfer price of a ‘task 2’ assignment increased 
to EUR 81,300, an increase of 46% compared to the ‘task 1’ average. One possible 
explanation for the rise could be a more intensive intervention delivery. As mentioned 
above, our analysis at the sub-module intervention level does not indicate any significant 
change between 'task 1’ and 'task 2' programmes. A comparison of both assignments reveals 
that the average overhead percentage did not reduce overtime - on the contraiy, a slight 
increase can even be discerned.' 42 

Considering the profit per delivered company, it is good to keep in mind that one of the 
drivers for these profits are the exaggerated mark-ups for overhead that came with the new 
cost price System. The average mark-up introduced in the cost prices for task 2 ECPs was 31% 
(10.5% programme management and 20.5% programme overhead), whereas the figure 
indicates the average to be 23%. Another profit driver was the budget rise of 17% discussed 
in the previous section. 

139 Capital ceiling set by the Ministry of Finance. 

' 4 ° By introducing a cap on equity accumulation, excess profits were returned to MFA, followed by 

agreements between dient and CBI to carry through rebates on the sales price. This example indicates 
that CBI has not been able to set an accurate cost price, despite being pushed by MFA’s accounting and 
control unit. 

141 This is about 2 % on average per year, which reflects the annual inflation. 

1,2 Overhead has been expressed as direct and indirect personnel costs, general fund, CBI innovation, 
evaluation and unforeseen. Source: CBI. 
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One of the determinants of overhead is the programme drop-out rate, being one of the 
indicators of its efficiency. Ideally, effective targeting should reduce the drop-out rate. 

The table shows that the ‘delivery rate’, as a reciprocal of the drop-out rate, varies across 
ECPs. Furthermore, costs associated with drop-outs (and those rejected after programme 
admission) have been expressed in terms of total ECP expenses. Despite the fact that the 
costs of drop-outs are limited compared to total costs, the number of drop-outs influences 
the costs per competent exporter significantly. 

The drop-out rates vary strongly across the four case study countries (Table 19). The figures 
indicate that South Africa is the country with the highest rate of companies that left the 
programme, almost twice the average drop-out rate. The last column seems to suggest that 
higher drop-out rates are more likely in lower-middle-income countries. 


Table 19 Drop-out rates per country 

Country 

Companies 

delivered 

Drop-outs 

Country 

incomegroup 

Bosnia and Herzegovina 

2 

6 

UMC 

Kenya 

2 

2 

LIC 

Indonesia 

8 

3 

LMC 

Philippines 

10 

3 

LMC 

South Africa 

22 

6 

UMC 

Peru 

8 

2 

LMC 

Thailand 

5 

1 

UMC 

Colombia 

44 

8 

UMC 

Macedonia 

23 

4 

UMC 

Ecuador 

18 

3 

UMC 

Serbia 

7 

1 

UMC 

India 

164 

23 

LMC 

Pakistan 

22 

3 

LMC 

Egypt 

31 

4 

UMC 

Ghana 

1 

0 

LMC 

Sri Lanka 

10 

0 

LMC 

Armenia 

2 

0 

LMC 

Tunisia 

8 

0 

UMC 

Vietnam 

1 

0 

LMC 

Honduras 

3 

0 

LMC 

Jordan 

8 

0 

UMC 

El Salvador 

1 

0 

LMC 

Tanzania 

1 

0 

LIC 

Bolivia 

4 

0 

LMC 

Uganda 

4 

0 

LIC 

Bangladesh 

1 

0 

LIC 


Bo/d indicates the case study countries. 
Source: CBI/SAGE. 
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5.3.1 Cost-effectiveness 

Translating the fmdings from survey and case studies to the ECP population under review 
enables us to develop an understanding of the number of assisted companies that benefited 
from the programme interventions, which can be used to determine the programme’s 
cost-effectiveness. It is important to note that enterprises may drop out at various levels in 
the result chain, so that the number of companies that actually benefit might be smaller 
than the number that might potentially do so. A reconstruction has been made for all ECPs 
based on the evaluation fmdings (Table 20). 

The table describes the various phases, starting from the selection into the programme by 
CBI. Of the 479 companies that got selected into the programme, 14% dropped out during 
the intervention, leaving 410 competent exporters. 

How does this translate into the cost-effectiveness of the programme? The total ECP 
portfolio accounted for EUR 24 million, or on average EUR 58,000 per competent exporten 
If the beneficiary group was to be limited to the number of companies that achieved an 
effect, or even a sizeable effect, the average cost per company would increase substantially. 


Table 20 ECP population effects (simulated) 


Phase 

Pipeline 

Number 

of ECPs 

Share of initial 

selected 

Share of competent 
exporters 

1 

selection into the programme 

479 

100 % 


2 

drop-outs 

69 

14% 


3 

competent exporters 

410 

86 % 

100 % 

4 

effect (all sizes) 

180 

38% 

44% 

5 

effect (2 EUR 145,000) 

62 

13% 

15% 

6 

sustainability 

30 

6 % 

7% 

7 

substitution 

15 

3% 

4% 


Source: IOB. 


Another way of assessing a programme’s cost-effectiveness is the return on investment. 

CBI uses this defmition as a Standard for assessing the impact, which is defmed as the total 
achieved exports to EU markets measured one year after completion of the programme. 
The export turnover is measured from the start of the intervention until one year after 
completion of the programme. 143 

Applying this defmition to the survey results, programme costs can be set against the 
additional exports (instead of total exports achieved). According to this defmition, export 
turnover is the sum of additional exports over the years 2006-2012 plus the sum of baseline 
export turnover during these years. Please note that we use a longer time frame than the 


CBI theory of change and indicators, 13 November 2013. 
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five year time frame used by CBI, which enables us to record effects more than one year after 
completion of the programme. 

IOB estimates that additional exports during the period 2006-2012, extrapolated for the 
population of CBI-assisted companies (13 ECPs), amount to EUR 29 million to the EU and EUR 
31.5 million to non-EU countries. 144 Given the average programme costs per competent 
company of EUR 58,000, one could state that for eveiy euro spent by the programme, 
additional exports achieved amounts of EUR 1.2 (EU) and EUR 1.3 (non-EU). When, for 
comparative reasons, the review is limited to 2006-2011, the additional exports achieved 
would be reduced to EUR 0.49 (EU) and EUR 0.53 (non-EU) for every euro spent. 

In conclusion, when defined according to the return on investment of attributable 
programme effects it can be stated that these have not been cost-effective for the 13 ECPs 
under evaluation, even when looking at a longer time span. Also, when looking at total 
additional (EU and non-EU) export development, according to CBI’s internal standards, we 
conclude that the programme was not cost-effective. This is even more the case when, instead 
making use of CBI’s own internal rate of return definition, the attributed value added or 
proflts are taken as relevant indicators for determining cost-effectiveness. 


5.4 Benchmark 

Apart from discussing efficiency based on internal organisational and programme data, 
we collected and compared benchmark data from external trade promotion organisations.' 45 
The goal of the external benchmark is to assess the efficiency of CBI’s services by comparing 
CBI’s cost structure to the cost structures of similar organisations. This evaluation 
concentrates on output and outcome efficiency, specifically for the export coaching 
programmes. In other words, the ECP costs have been related to output, i.e. numbers of 
assisted exporters, and outcome, i.e. additional exports realised. 

Costs and cost drivers 

Table 21 presents the total annual cost of the activities that most resemble CBI’s export 
coaching programmes. Overhead is defined as all expenses for secondary (or support) 
processes, such as general management, fmance, marketing, and human resources 
management. CBI’s breakdown of primaiy costs (‘ programmakosten ’) and secondary costs 
(‘opslagen’/ Voorbelastingen’) has been used. Programme management is seen as part of the 
primary process and thus is not included in overhead. In SIPPO’s annual activity reports, 
programme, project, and other organisational costs are spedfied.' 46 

’ 44 It should be noted that the ECP evaluation perspective used here differs from that used by CBI. Instead of 
evaluatingall companies participating in the same ECP, the subject of the current evaluation is ECP- 
assisted companies in a specifïc country. 

’ 45 See Annex 12 for a methodological description of the benchmark. 

146 According to a COMO consultant who studied SIPPO’s accounting system in 2010 , programme management is 
accounted for in programme costs, which is also the case atCBI.Therefore.the breakdown in the annual reports 
is considered useful for such a comparison. For TDC, the average overhead percentage has been calculated on 
the basis of two annual reports, and this has been added to the costs for the export coaching activities (‘Renforcer 
les capaatés et l'accès au matché des producteurs du Sud’and ‘Encourager les échanges commerciaux Sud-Sud’). 
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Export coachingcostsincl. overhead EUR 11,414,373 EUR861.099 EUR 5,997,946 

Export coachingcostsexcl. overhead EUR 7,987,099 EUR 672,734 EUR 5,521,132 

Overhead to expenses 30% 22% 8% 

Source: IOB. 

The cost of export coaching is a net cost, meaning that contribution fees have been 
accounted for. The share of contributions by beneficiaries is divided as follows: CBI (6%), 
the TDC (20%) and SIPPO (7%). 

To reflect the scale of operations, and thus the potential for economies of scale for each 
TPO, we have gathered information about the share of export coaching activities in the 
organisation. We selected TPOs for comparison as they have separate financial accounts, 
which allows us to do this at the TPO level. It is important to note that all three TPOs are in 
fact part of a larger organisation. TDC is a part of the Belgian Development Agency (BTC). 147 
The annual accounts show that TDC uses BTC's infrastructure, and pays for it. SIPPO has 
over the pastyears been integrated into Switzerland Global Enterprise, just as CBI is 
currently being integrated into RVO. 

The table clearly shows that CBI is the largest organisation in terms of turnover, while SIPPO 
focuses most on export coaching, with all activities plausibly targeted at increasing its 
beneficiaries' exports. Although SIPPO apparently began to shift its focus onto more 
systemic interventions in 2012, this shift was notyet evident in thatyear’s financial report. 


Table 22 Export coaching share in 


Total annual turnover (IPO level) 

Export coaching share in organisation's 
turnover 
Source: IOB. 

Cost differences can also be related to programme characteristics. In terms of programme 
duration and content, CBI and SIPPO are quite similar (Table 23). The Standard programme 
duration in both organisations is three to fouryears. While evaluation and administrative 
(preparation and closing) activities are not fully included, the part of the programmes in 
which companies actively participate is similar. Also comparable is the number of visits to 
trade fairs: companies participate in a maximum of three fairs, and have to make a higher 

147 TDC is a part of BTC, which is a much larger organisation than CBI. BTC has offices and personnel both 
in Belgium and abroad, and has a turnover of around EUR 229 million (2012 Annual Report). The cost 
structure for the overall organisation has been made transparent in its annual reports. But as TDC is 
such a small part of BTC (+1 %), we do not believe the overall cost structure to be relevant for our 
comparison with CBI. 


total turnover (average 2010-2012) 

CBI TDC SIPPO 


EUR 22,516,783 EUR 1,968,769 EUR6,047,292 

51% 44% 100% 
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contribution for each subsequent fair. The actual amount of trade fair visits per company is 
unknown for SIPPO, as it does not register drop-outs. 


Table 23 Programme characteristics (average 2010-2012) 

CBI TDC SIPPO 


Duration 4.4years 

Estimated number of 597 

beneficiary companies 

peryear 

Average number of 32 

beneficiary companies 
per programme 

Amount of sectors 19 

targeted peryear 

Main sectors targeted Industrial products 
(e.g. medical devices, 
engineering products, 
personal protection 
equipment), 
consumer products 
(e.g. clothing, home 
decoration and 
accessories), services 
(e.g. IT outsourcing), 
agriculture, food and 
forestry (e.g. food 
ingredients). 


n.a. 3years 

(max. 4 years) 

843 'individus' 1,000 

+ 27 'groupes' 

n.a. 65 


2 15 

Agriculture (80%), Food (fish and seafood, 
handicrafts. fruit and vegetables, 

natural ingredients, 
bio-products), non-food 
(home textiles, alpaca, 
fashion, jewellery, 
handicraft, indoor 
furniture, outdoor 
furniture), technical 
products (metal proces¬ 
sing and mechanical 
engineering, software 
applications, technical 
wood), tourism. 


Source: IOB. 


I 99 


Whereas SIPPO and CBI offer multiple export-oriented services (such as individual coaching 
and collective visits to trade fairs) integrated into a single programme, TDC has notyet 
developed such an integrated programme. Financial and technical support to businesses 
thatwant to visita trade fair, forexample, is a separate activity from marketing support. 
TDC's programme does not have a predetermined duration, as it is fully tailor-made, based 
on participants’ own plan of action. 

The amount of personnel needed to execute programmes is not reported here, as we could 
not fmd any trustworthy data. The export coaching activities are performed largely by 
contractors, and the number of contractors has not been reported upon in any of the cases, 
nor has the size of their contracts. 

The CBI and SIPPO programmes differ quite a bit in terms of the number of companies 
participating in export coaching: SIPPO has twice as many companies. CBI targets more 
sectors in any given year than SIPPO, i.e. 19 as opposed to 15. 
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A special attribute of TDC is their exclusive focus on fair trade. Many of its activities involve 
promoting and certifying fair production methods. At CBI and SIPPO, fair trade is neither a 
selection criterion nor the main focus of the export promotion activities, although SIPPO 
currently has one programme that deals with responsible tourism and CBI is working on 
integrating international corporate social responsibility criteria into its opera dons. 

Choice of sectors and products 

CBI and SIPPO have a comparable sector focus, which can be described as a combination of 
agricultural, industrial, and services industries, which includes both products intended for 
business-to-business and business-to-consumer trade. TDC stands out in this respect, with a 
clear focus on agricultural products (mostly commodities) and handicrafts. Unlike CBI and 
SIPPO, TDC does not work at all with industrial companies. In practice, ECP participants are 
mainly from the industrial and consumer goods sectors, and less so from agriculture. 

Geographical spread 

The number of countries in which export coaching programmes were implemented differs 
quite a bit between the three organisations. CBI has the broadest focus (29 countries on 
average) and TDC the narrowest (7 on average). The difference in geographical focus 
between SIPPO and CBI can be explained by the fact that SIPPO has had to work with a 
much shorter country list than CBI. Their sponsors (SECO and the MFA, respectively) only 
allow them to work in a predetermined set of countries. 


Geographical spread (average 2010-2012) 
I CBI 


Average amount of 


29 


14 


Countries targeted 

Afghanistan, Armenia, Bolivia, Ecuador, 

Albania, Bolivia, 


Bangladesh, Bolivia, Democratie 

Bosnia and 


Bosnia and Herzegovina, Republic of 

Herzegovina, 


Colombia, Cuba, Ecuador, Congo, Germany, 

Cambodia, 


Egypt, El Salvador, Kenya, Morocco, 

Colombia, Egypt, 


Ethiopia, Guatemala, Palestine, Peru, 

Ghana, Indonesia, 


Honduras, India, Rwanda, Senegal, 

Jordan, Kosovo, 


Indonesia, Jordan, Kenya, South Africa, 

Laos, Macedonia, 


Macedonia, Mali, Morocco, Tanzania, Uganda. 

Nepal, Peru, 


Pakistan, Peru, Philippines, 

Serbia, South 


Serbia, South Africa, Sri 

Africa, Ukraine, 


Lanka, Thailand, Tunisia, 

Uganda, Vietnam, Zambia. 

Vietnam. 


Source: IOB. 


Looking at the type of countries targeted, we note that sub-Saharan countries are over- 
represented at TDC (7 out of the 13 countries worked with in 2010-2012) and under- 
represented at CBI (5 out of 43) and SIPPO (2 out of 18). 




CBI’s efficiency 


Target group characteristics 

The evaluation of SIPPO and CBI has shed light on the types of business that enrol in export 
coaching programmes. It seems that CBI’s beneficiaries are somewhat larger in terms of the 
number of employees. Even so, SIPPO and CBI beneficiaries both fall within the limits of 
any defmition of SME. Both CBI and SIPPO’s beneficiary companies have considerable 
exporting experience, even to the European market. This suggests that these businesses are 
quite advanced, indeed on the higher end of the value chain, as SIPPO has put it. 

TDC, on the other hand, works mostly with small mainly agricultural producers 
(smallholders). Many of its activities aim to unite individual producers into cooperatives 
and integrate these cooperatives into newly developed value chains. 

Looking at the tables presented in the sections above, SIPPO is most likely to have the 
lowest cost per exporten SIPPO has twice as many participants in its sector programmes, 
suggesting lower start-up and evaluation costs per participant. In addition, these 
participants are probably located in far fewer countries, suggesting lower travel costs and 
time needed to visit them. 


5.5 Conclusions 

• The overhead needed to run programmes increased at a faster pace than the programme 
budget. 

• The creation of CBI as an independent agency did not make it more cost efficiënt. One 
possible explanation for this is that there was no MFA-induced incentive for CBI to 
lower its costs. 

• With the introduction of ex ante cost prices and the modular programme delivery, CBI 
has seen its turnover and profit rate rise, with profits increasing more rapidly than 
turnover. 

• MFA approved requests for budget increases, even though they were not always justified 
by CBI and lacked information about cost prices. 

• Our empirical analysis could not find any justification for CBI’s extra budget claims either, 
which supposedly resulted from the intensification of its interventions over the years. 

• CBI agreed to pay back accumulated excess Capital to MFA, even introducing discounts 
on its tariffs. This may create the wrong impression of an efficiently run operation, 
while in fact cost prices had been too high for too long. 

• Translating the number of companies that benefited from the programme interventions 
into cost-effectiveness, we conclude that the average cost per company is far above the 
Financial settlement per competent exporten 

• The benchmark indicates that CBI had the highest cost per exporten This was caused by 
high fixed costs per participant and high transaction costs as a result of its presence in a 
wide range of countries. 

• As an independent agency, CBI was not run efficiently, and the programme itself was 
not cost-effective either, because it did not optimally direct its investment of scarce 
resources towards the group of companies that needed it most. 
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6.i Introduction 

We will discuss the relevance of the BSOD programme by juxtaposing fmdings from the 
literature with CBI’s assistance rationale, curriculum, and implementation of the BSOD 
programme in Section 6.2. Section 6.3 presents an assessment of the effects of the BSOD 
programme, followed by a reflection on the efficiency of its implementation in Section 6.4. 


6.2 Relevance of the BSOD programme 

One challenge facing private sector development institutions is how to achieve operational 
scale when providing support to SMEs in developing countries. The problem is that SMEs 
are by nature small, diffuse and numerous. Supporting SMEs carries with it the inherent risk 
of the institution itself becoming too diffuse, as highlighted in a number of recent 
evaluations of development assistance to SMEs. One way through which institutions seek to 
increase the scale of impact of trade-related SME support is to focus on strengthening 
intermediary organisations, such as trade support institutions, as highlighted in a recent 
evaluation (ITC, 2013). 148 

In recent decades, stimulating national exports by offering information, fmance and 
technical assistance to individual firms via trade promoting organisations has received 
broader attention. The number of TPOs that represent national interests has tripled over the 
last two decades (Lederman et al., 2010). The literature tells us that the types of service 
offered by these TPOs seem - at least on the surface - quite similar across countries and can 
broadly be divided into three categories: 

1. market research and publications (offering firm, sector or countiy-specific market 
information on tariffs, standards, trends, opportunities, and competition, for example); 

2. training and technical assistance (training and advice on export processes, the 
regulatory environment, legal issues, logistics, marketing, and developing export 
business plans); and 

3. market entry support (facilitating and (co-)financing trade fair participation, trade 
missions, and matchmaking activities). 

Theoretically, these services could promote export by reducing information asymmetries 
and transaction costs, and by giving firms incentives to invest in entering new markets. 
However, in the past there has been a great deal of criticism on the functioning of TPOs in 
developing countries. Hogan et al. (1991) argued that TPOs were generally ineffective 
because of inadequate capacity, insufficiënt customer orientation, heavy government 
involvement, and a lack of long-term support to companies. Since then much has changed 
(arguably in response to these criticisms). Nevertheless, the question of the additionality of 
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148 WTO, ‘Connecting developing country SMEs to global value chains'; joint communication from the 
International Trade Centre and the WTO secretariat, September 2014. 
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donor programmes in specific fields has become more pressing, and requires donors to 
become even more selective choosing where and who to support. 

CBI's management has shown a greater interest in expanding the BSOD portfolio in the last 
decade. It assumed that this product could contribute more effectively and efficiently to 
CBI's greater goal, i.e. that a larger group of companies could be reached for less cost per 
company compared to the ECP programme. While the initial focus was entirely on 
reinforcing local business support organisations, since20ii, CBI hasattempted to create 
more synergy between supporting BSOs and companies and is trying to achieve this in a 
more coherent and structured mannen 

BSOs need qualified and well-trained staffwho have the competences to provide relevant, 
digestible, practical, and tailor-made EU export marketing and management services to 
their member clients. These services include skills pertaining to market research and trade 
promotion techniques for BSO staff so they can identify different sectors and niches in new 
EU markets. And they may also include services to individual exporting companies, like 
export audits, market research, or providing information on market requirements. 

Box 8 Typical BSOD deliverables 

i04| 

The following list contains examples of results that are typically generated by BSOD 
programmes: 

• collaboration models and agreed task divisions with other export service 
providers and actors in export value chains; 

• improved organisational vision, policies, strategy, and structure enabling 
adequate European market-related export service delivery; 

• a quality assurance System (including manuals, established dient feedback 
processes, internal quality control procedures, etc.) for European market-related 
export services; 

• business processes for (European market-related) service delivery; 

• marketing and communication of the market proposition of BSOs to their 
dientele and stakeholders; 

• a network of local external experts under the umbrella of the BSO; 

• enhanced competences of BSO staff regarding identifying and transferring 
European market opportunities to exporters; 

• enhanced competences of BSO staff to deliver quality services for export value 
chains and exporters to the European market; and 

• new (European market-related) products and services developed by BSOs. 


Source: CBI. 
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Given the heterogeneity of BSOs, the relevance of the technical assistance depends very 
much on the mandate and competence of the beneficiary. For example, CBI supported the 
Indian BSO TEDO, which was having difficulty expanding its activities as the organisation’s 
headcount was limited. As a result, they had to build a small network of external experts 
who supported them in their programmes. The feasibility report mentions strong signals 
that there was a lack of strategie coherence and professionalism at TEDO. Therefore, the 
technical assistance of the programme was highly relevant to TEDO’s mandate. 

The evaluation of SIPPO also indicates that the quality of cooperation is not uniform, since 
BSOs are weak in many countries. Their fmancing is not secure and their choice of 
personnel is often politically motivated, and therefore the work relationship between CBI 
and beneficiaries depends on individuals, not on structures. 148 For that reason, SIPPO views 
joint strategie planning as wishful thinking and usually unrealistic. 


6.3 Effects 

According to the literature reviewed, the capacity and internal organisation ofTPOs are 
important determinants of their effectiveness (Lederman et al., 2010). Lederman et al. 
demonstrate that TPOs perform well when they are publidy funded but also at least partially 
managed by the private sector. Apart from this paper, however, there is little rigorous 
evidence on what type of internal organisation is important and whether building the 
capacity TPOs is an effective strategy. The reason for the lack of good studies, according to a 
review by SIDA (2013), is the absence of objective indicators that can measure increased 
capacity (e.g. how to measure whether the training of staff led to improved services) and the 
difficulty in attributing the observed changes at a company or country level to capacity 
building activities at an organisation that operates on a national or regional level. 150 

6.3.1 Challenges of measuring capacity development effects 

The exact results that a BSOD programme will achieve depends on the particular 
organisational needs of the BSO, as well as the country and/or sector needs. 151 One year after 
a programme ends, an independent evaluation is conducted to assess the extern to which 
the programme’s purpose/effect and objective/impact have been achieved. The evaluation 
uses quality indicators that rate a programme as ‘sufficiënt’ if (a) 80% of the competences 
have been transferred, (b) 60% of the transferred competences are being used, and (c) 60% 
of the clients of the services provided by the BSOs agree that they are receiving better service 
provision than they did prior to the support provided by CBI. 


One important learning point for the BSOD programme at the Indian BSO TEDO is the implementation 
of the strategie planning module. TEDO worked on projects and was relying too heavily on external 
funding. As result, it could not grow as an independent EU export-supporting entity within its mother 
company, as initially expected. 

150 Seealso IOB evaluation no. 336 ‘Facilitating resourcefulness: Evaluation of Dutch support to capacity development’. 

151 These needs are assessed during the programme’s preparatory phase. In this stage, one or more of the 
four available BSOD modules are considered for implementation: strategie planning for the EU, export 
marketing and management, market information, and market development. 


1105 


Aided trade 


106 | 


Several BSOD evaluators indicated that measuring such a quality Standard is a great 
challenge, since interviewees have little understanding of how to interpret indicators such 
as ‘transfer of competences' or ‘use of the capacity’, nor are they able to provide a sound 
opinion on the ‘effect of the use of capacity’. For example, one of the evaluators stated that 
60% to 80% of BSO staff expected their job performance towards exporters to improve after 
training. One should also note, however, that the reported figures should be read as 
anticipated rather than actual impact. 

These quality standards set by MFA and CBI seem to be too general and hard to measure 
accurately. In other words, assessing effects or impact is hindered by the absence of 
objective indicators that can measure increased capacity. Moreover, since the effects of 
organisational changes are less likely to take place immediately, but might manifest only 
after several years, they will not necessarily be captured during the regular evaluations that 
take place one year after the intervention has terminated. To make it even more complex, 
the fact that the project is normally implemented for several years increases the possibility 
that the learners/participants fail to accurately recall their experiences. They might still use 
knowledge, skills, and methodologies provided by CBI, but may be unable to name these 
things due to the fading memory over time. There is another, more culturally induced bias, 
also highlighted in some BSOD evaluations, which tells us that recipients are more inclined 
to give socially desirable answers. 152 

Aware of the limits of indicators, and the distorting effects mentioned above, CBI has 
experimented with other assessment methodologies, which did not lead to any radically 
different approach. One example is the 5C model, proposed by one of the evaluating 
bodies, which suggests a more objective and process-oriented evaluation perspective, as 
opposed to a task and result-oriented one. ‘It attempts to make profound changes in the 
organisation that could take a longer time than the one established for the BSOD 
programme, but also, it could aim for a broader objective that may not necessarily or easily 
translate into export gains in the short run’. 153 

Evaluators indicate they were limited in their research by a more general problem, namely 
the lack of reliable data. 154 No data has been specifically collected to establish a baseline or 
end-line situation related to the BSOD, or to measure cliënt satisfaction. Moreover, 
evaluation outcomes are predominantly based on self-perception, and the measurement of 
BSO clients’ satisfaction is not an integral part of evaluations. 155 With some notable 
exceptions, no quantitative data is available to draw conclusions on the programme’s 
contribution to increased exports. The lack of data also suggests that the programme 

,5! For example, one of the evaluators mentions that the recipiënt organisations’ staff is rather obedient 
and in need of a more assertive mindset, also vis-a-vis CBI. 

153 Efece & asociados s.r.l., 2013. 

154 Because data was unavailable, or when available, conclusions are based on small sample sizes. The 
smaller the sample size, the larger the bias. Efficiency could not be assessed because of a lack of 
concrete fïnancial data due to the change in CBI’s (accounting) System. 

155 To evaluate the impact of the programme, CBI could do a (qualitative and/or quantitative) survey with 
SMEs and other BSOs at the beginning and the end of the programme. 
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management lacks the information to monitor and steer programmes in the desired 
direction.' 56 

The regular BSOD evaluations on behalf of CBI indicate that the assessment of the quality 
criteria could not be as specific as expected. Also, the limited availability of data to evaluate 
the impact of the programme on SMEs/exporters leads to descriptive rather than evaluative 
conclusions. 

6.3.2 Findings 

Our review included twenty-one of the fifty-six BSOD modules: three in Bolivia, one in 
Ecuador, three in India, four in Indonesia, three in Peru, three in the Philippines, and four 
in Vietnam. 

Organisational capacity 

A review on behalf of CBI concluded that in general evaluators were moderately positive 
regarding achievements at the result and effect levels (Triodos, 2009). The BSO staff 
members interviewed were satisfied with the programmes and indicated that many of the 
acquired tools and skills were being applied in practice. As the report on the Investment and 
Trade Promotion Centre (ITPC) in Vietnam puts it: ‘The training introduced content and 
supported the restructuring process of ITPC. Staff received content to get involved in the 
work, while in the past they had concentrated on administration’. 157 As a result, the 
professional organisation and service packages of ITPC improved. This has boosted the 
organisation’s confidence and status, making it more attractive for exporters to use their 
services. 158 CBI also helped to identify and develop new export products, though it remains 
unclear to what extern these products will actually be marketed on a large scale. According 
to some participants, marketing these products is still high-hanging fruit as the road from 
production to export is long. 

One aspect of the export marketing and management module (EMM) has been the inclusion 
of local consultants and service providers, through a ‘train the trainer’ (TTT) programme. 
The anticipated multiplier-effect of trained trainers remained below expectations because 
SME-level data measuring dient satisfaction is lacking, as well as information on export 
increases. Of course, there are exceptions, such as the example of Indian consultants 
trained by the BSOD programme, who managed to start their own commercially based 

156 For example, it is difficult to measure the effect and impact of CBI training courses when former 
participants of these courses cannot be tracked. A good quality database would make it relatively easy 
to develop a cost-effective way of measuring effects and impact over time. 

157 Thanks to CBI support, the organisation is now better at designing its curriculum and syllabus, 
approaching customers, and managing customer relationships. In the Philippines, at the start of the 
BSOD programme, Philexport Cebu did not have a market information service at all. 

158 Bringing more structure and ‘content’ to the work of BSOs seems to be an important asset of the BSOD 
programme. From this perspective it is good to understand the positive reactions of BSOs while 
implementing a market intelligence plug-in on the organisations’ website, since it creates a new and 
improved service offer for the organisation. On the contrary, companies themselves, as ultimate 
beneficiaries of market information services are more diverse in their assessment of these services, 
especially more experienced organisations expressing often critical opinions. 
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advisory service, proving that opportunities can be converted once the cliënt values the 
expertise and is willing to pay for such services. 159 

Increased exports 

As indicated, no quantitative data is available to draw conclusions about the programme's 
contribution to increased exports. Nevertheless, in qualitative terms, various authors agree 
that the direct or indirect effect of the projects on the capacity of SMEs to export is minor. A 
review by Triodos concludes that trained BSO staff, local consultants, and exporters expect 
training to have a positive effect in terms of contributing to increased exports. Nevertheless, 
six months after training there appeared to be no effect on export development, and even 
though exporters’ knowledge increased, new product development remained limited. 160 
In other words, this leaves the impression that workshops as stand-alone activities cannot 
contribute significantly to export, and that such expectations are in fact unrealistic and 
exaggerated. Indeed, since most of the anecdotal and observable evidence does not point 
towards the claimed effect, stronger facts or metrics that would underpin the opposite, i.e. 
that the BSOD programme would have a positive indirect effect on exports, still need to be 
presented. Moreover, these evaluations also indicate a shortfall of the programme, namely 
that BSOs have not been able to tackle other barriers that prevent SMEs from exporting, 
such as the business community's lack of English language skills or the small size of 
companies. 

6.3.3 Explanations 

This section touches on some of the reasons why the objectives of the BSOD programme 
were not achieved, and in doing so distinguishes between the programme design, the 
programme implementation, and target group characteristics. 

A lack of clear objectives and focus 

Business support organisations with a strong sector focus seem to have benefited most 
from organisational development support. This is in line with the conclusion in the 
evaluation of CBI’s Swiss counterpart SIPPO. 161 Cooperation seems to improve with 
sector-oriented partners, whereas the chances of a successful intervention are more limited 
when working with organisations that have a broad mandate and political mission. Large 
organisations are often bureaucratie and less sensitive to change. They often have a specific 
organisational structure, mission, Vision, and co re values that cannot be changed easily. 
There is also a difference between public BSOs, 162 which rely heavily on government funds 


159 In the Philippines, for example, some of the local consultants were being hired for their expertise by 
companies or local government, without intervention or mediabon by Philexport or CBI. The 
consultants are too expensive for Philexport, but got paid by CBI through the Export Diversifïcation 
module. 

160 For example, the impact of the BSOD programme at IMAFED has been modest, suggesting that none of 
the SMEs that participated in the BSOD programme conducted any export activities. 

161 For example, SIPPO’s support of the association of the textile sector in Vietnam (Vietcraft) proved to be 
much more reliable and concrete compared to Vietrade, a non-sector oriented governmental agency. 

,6z For example DGINIED (Indonesia) and HCACS (Vietnam). 
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and for that reason are not inherently motivated to expand their cliënt base vis-a-vis private 
sector organisations that are in need of clients and thus willing to pay for their services. 163 

A more detailed analysis of the context in which BSOs operate could help to more accurately 
establish these organisations' actual needs. According to the evaluator of supported BSODs 
in Latin America, a continent where business support has a long history, the assessment of 
the perceived priorities by the BSO’s top executives should have been more detailed and 
carried out in tandem with an assessment of the BSO's needs. That would have it made it 
possible to select a core process for spreading specific BSOD interventions throughout the 
organisation. 

CBI's predefmed modular intervention approach does not always give enough attention to 
the internal and external dynamics and politics of the BSOs. In some cases, the programme 
did not understand, interpret, or properly address the environment in which the BSOs were 
operating. In such cases, the cost of not having a precise understanding of the BSO's context 
resulted in growing internal tensions as the programme progressed. 

BSOs could have benefited from more sensitivity on CBI’s part regarding the local project 
setting, especially when BSOs are part of a network. Apparently reports that date back to the 
formulation of the BSOD intervention in South America already mention the problem of 
cultural differences within BSO networks, and the fear of losing autonomy when favouring 
a more empowered beneficiaiy. The fundamental question then becomes how CBI can 
develop a better understanding of these potential constraints, and how to address them. 164 

Supply driven, lack of market demand 

The development of new European market-related products and services is an example of a 
deliverable from the BSOD programme. But as the example of the BSO CADEXCO’s HACCP 
certification services shows, a newly developed service that does not appeal to the BSO’s 
clients, or is priced unrealistically vis-a-vis real market prices, is condemned to disappear. 165 
In cases such as this, all the efforts by CBI, the external experts, and the BSO have no 
tangible impact as a result. A very conventional way to solve this is the same way that any 
private company would, namely by determining whether the effort to try to sell a new 
product is justifiable. This type of analysis should guide the areas of intervention based on a 
thorough study of demand conditions for such services. 


163 For example, the publicly funded HCACS clients are beneficiaries of local government-supported 
services and the BSO is the budget allocator; hence this does not represent a market customer-oriented 
process. 

164 To assist competing BSOs at the same time can be quite frustrating. This is not exclusive of CAN EB, it 
was also the case in ADEX and the Lima Chamber of Commerce in Peru; the members of the network 
were competing for the same clientele with very similar kinds of services. 

163 Based on a pilot project in the natural ingredients sector it was estimated that, on average, 48 

companies would demand HACCP certification each year after CADEXCO and a local consultancy with 
experience in this sector, started offering the service. One year after programme closure, no company 
had been certified and only one has expressed real interest. 
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The BSOs have local consultants participating in their events, training sessions, and 
consultancy assignments. As mentioned, the effect of this group of trained consultants on 
the capacity of enterprises to export should be considered minor. Not surprisingly, many of 
the trained consultants are somehow frustrated by the promise that was made after they 
had participated and passed the training (Triodos, 2009). They expected further initiatives to 
unfold and also expected to participate in projects or programmes. However, there has 
hardly been any follow-up or communication following the TTT programme, and graduates 
indicated that they feit the training was too short to give them enough ammunition to face 
the industry. It is therefore recommended that future capacity building interventions 
should focus more on sectors instead of general export management only. 

Other evaluations mention the suggestion by local experts that the level of training be 
customised, which could have been achieved by including local sector experts to deliver 
certain parts of the EMM training. Including local experts is a potential problem as there is a 
risk of inconsistent programme deliveiy (e.g. in the case of IETC from Indonesia) since 
different local experts might not share the same view of a given matter. However, the way 
they deliver services to local SMEs should be consistent. This helps to assure the quality of 
the programme, protect the image of the donor and BSOs, and protect learners and 
participants from confusion. 

Ownership, Ieadership and commitment 

As the case of Philexport shows, ownership and Ieadership commitment are preconditions 
for a programme’s success. In this specific case the director was already familiar with CB 1 
and had lived in the Netherlands for almost 20 years. Moreover, the organisation already 
had a clear picture ofits future strategy and plans, even before the arrival ofCBI. 

Allocating time efficiently by beneficiaiy staff seems to have been problematic. Each CBI 
mission expert delivers technical assistance for only a couple of days, but of course staff 
cannot always leave their daily job assignments to attend BSOD activities. This has limited 
the transfer of competences. 

In India the transfer of competences to just six TEDO coördinators, who were also 
responsible for other jobs at the mother institute, resulted in only temporary commitment 
towards the jobs assigned in TEDO. From this perspective it might have been more effective 
to offer the strategie planning (SP) module at the level of the mother institute. 

Another aspect of ownership is the high staff turnover rate at supported organisations, or as 
one of the evaluation reports phrased it: ‘high rotation is the enemy of sustained results’. 
This is a general problem, though at the same time a normal situation that occurs in cases 
where people are trained, and invest in themselves, which could also generate spillover 
effects. Nevertheless, the general criticism made by these evaluators is that the programme’s 
design could have paid more attention to sustaining the effects ofits deliverables. 
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This criticism was also voiced with regard to CBI’s HRD programme. The MASTER/MAR and 
FAME studies state that the use of newly acquired competences becomes less obvious at the 
BSO level. This is due to the fact that some of CBI’s seminars are predominantly an 
investment in individual development rather than in organisational development. So even 
though individual participants benefit from the training, the knowledge and tools obtained 
in the training are not always spread to others in the same organisation. Some authors 
argue that in order to strengthen effects at the organisational level, more follow-up is 
needed. However, both the FAME and the MASTER/MAR evaluations suggest that follow-up 
is not always an integral part of CBI’s training programmes. 

Last but not least, the English language proficiency of beneficiary staff has been mentioned 
as a problem in various interventions. Given the nature of the activities, this aspect should 
have played a major role in the selection of beneficiaries and staff. In some cases, CBI 
accommodated this problem by having the curriculum translated. The key question here is 
whether programme management should have made such a compromise at all, since 
English language proficiency is one of the programme’s admittance criteria. 

Limited additionality? 

Since CBI beneficiaries are used to working with a number of international partners, such as 
- in the case of IETC, for example - the Pacific Resource Exchange Centre in Japan, the Trade 
Facilitation Office in Canada, the Canada Indonesia Private Sector Enterprise Development 
Project, the Japan External Trade Organization and the International Trade Centre, effective 
coordination between service providers is often lacking, which leads to a suboptimal task 
division. 

In Indonesia, according to the evaluation report, CBI’s intervention via an SP module played 
a vety modest role in organisational change, from NAFED to DGNED. Indeed, NAFED’s 
organisational structure, Vision and mission had already been proposed by the Japan 
International Cooperation Agency (JICA). CBI’s role here was to provide a second opinion, 
and to align the proposed structure, Vision and mission with CBI’s policy to improve export 
performances to EU markets. 

The additionality is also questionable in the example of Philexport, which already had a 
clear strategie plan prepared before CBI arrived on the scene. In this case, the contribution 
boiled down to accelerating an already ongoing process. 

6.3.4 Sustainability 

The evaluation of CBI’s Swiss counterpart, SIPPO, reveals the weaknesses of its current 
approach in strengthening the export-related capacities of BSOs. These would also affect the 
sustainability of SIPPO’s interventions, as lasting changes in this field have not been 
achieved yet. Such a result would occur, for example, if a BSO that was not actively 
supporting or capable of supporting local companies to participate in international fairs 
learned to do so by cooperating with SIPPO and went on to offer this service at a reasonable 
quality on its own. 
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Some of the evaluations provide some evidence of self-sustaining BSO services, for example 
at Philexport. In this case the implementation of the MI module created a more or less 
self-sustaining service, where the increase in membership fees allowed payment for the 
salary of a market analyst. 

As the case studies show, CBI is said to have contributed to the development of business 
support operations even when no official BSOD programme had been implemented, and 
despite not being a formal part of ECPs. This happened in particular through partnerships 
that CBI forged with organisations like Expolink and FTTC in order to organise Iocal events. 
These organisations have stated that they still benefit from this capacity development. By 
working through the national export promotion organisation Proexport, CBI contributed to 
export promotion in Colombia. 

In the case of local consultants receiving training, twenty-five local consultants were trained 
by CBI experts in the EMM module atTEDO in India. Each consultant was assigned to assist 
four to five companies to develop an export marketing and strategy plan. As the BSO had 
Financial resources at its disposal from the government, the local consultants were paid for 
their Consulting services at the pilot SMEs. After the BSOD programme, some local experts 
created their own consortium to provide export-related services to Indian SMEs. 
Nevertheless, the evaluators reported cases in which the actual number of job opportunities 
available to the TTT graduates was lower than was suggested at the beginning of the 
programme, so they needed to pursue their careers elsewhere. 

From the perspective of the SMEs, the ultimate beneficiaries of BSOs, the sustainability of 
the programme depends on how the transferred competences are used to support SMEs in 
their efforts to export to the EU. Indeed, the local experts play an important role in 
sustaining the use of EMM and MI competences. Therefore, BSOs can play a key role in 
facilitating and mobilising the network of local experts to promote export in addition to 
each expert's proactivity, and most likely seem to have had an impact. 

In the majority of cases, the programme's focus on sustainability could have been better 
conceived. Evaluators recommend investing a large part of every BSOD programme in 
sustainability issues. This could start by developing a business plan at the beginning of the 
programme. Close to the end of the programme, an ‘exit strategy’ could boost the 
counterpart’s trust or confidence. The exit strategy should also be directed towards the 
stakeholders, who will need to have faith in the BSO without CBI’s support. CBI could have a 
monitoring and facilitating role in this phase. 166 Moreover, CBI should have a good 
understanding of the BSO’s Financial management. There should also be a good 
understanding of the market's willingness to pay for a given service. 


166 One proposed idea is gradually losing traction, namely the idea of facilitating one or two conferences 
with the managementteam and stakeholders a year after the end of the programme. In the case of 
Philexport, it is a good sign that MoUs were signed with donors that identifïed Philexport Cebu as 
preferred supplier of Ml and training and coaching services, thereby enhancing service delivery after 
termination of the BSOD intervention. 
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6.4 Programme efficiency 

This section describes the efficiency of the BSOD programme, based on what is known from 
currently available evaluations. The theme is analysed both from the institutional 
perspective (sub-section 6.4.1) as well as the project perspective (sub-section 6.4.2). 

6.4.1 Institutional perspective 

As the assessment of the effectiveness of the BSOD programme has shown, CBI is often not 
the only contributing donor. The key question for the BSOD programme, then, is how to 
complement and synergise with other projects to get maximum value out of the 
investments. 

As described in the evaluation of the Indonesian big BSO NAFED, this organisation also 
received assistance from other international partners, such as the Japan International 
Cooperation Agency, the Swiss Import Promotion Programme, the Hong KongTrade 
Development Council, as well as other international cooperation organisations. The 
evaluator concluded that providing complementary assistance can be only effective and 
efficiënt if the BSO is strongly committed to allocating CBI-trained staff to the right position 
after its organisational change. This change, incidentally, was brought about by the JICA 
programme. Therefore, the BSOD capacity building component should have been launched 
after NAFED was restructured into DGNED. Moreover, the report also suggested that if CBI 
had focused on the EMM module instead of the strategie planning module, it might have 
increased efficiency. 

In the case of the Indian beneficiaiy TEDO, the intervention, though relevant and aligned 
with government policies, would have been more effective and efficiënt if the mother 
organisation had been targeted instead of TEDO. 

6.4.2 Programme perspective 

Evaluating the efficiency of BSOD programmes at the project level is seriously hampered by 
a lack of documentation in the project files. The systematic lack of planned Financial 
resources in each of the modules, in particular, as well as the budget have not been an 
integral part of the documentation. A detailed budget overview would have clarified the 
planned budget allocation for each of the projects. Furthermore, an assessment of the spent 
budget can only be made if there is information available on the outputs and their quality. 
Here too, we must conclude that insufficiënt information is available to review the BSOD 
evaluations. 
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Table 25 Assessment of BSOD programme modules 


internal/external 

country 

BSO 

status 

module 

relevance 

effect 

efficiency 

expenses(in EUR) 

over- / underspending 

output 

CBI 

Vietnam 

ITPC 

public 

SP 

+ 

+/- 

*/- 

393.000 

+4.5% 


CBI 


ITPC 


EMM 

+ 

+/- 

-- 

549.000 

+29% 

40-50 advisors 
trained 

CBI 


ITPC 


Ml 

+ 

+/- 

-/+ 

492.000 

+ 12% 

portal 

launched; 

Ml strategy 
development 

CBI 

Philippines 

Philexport 

Cebu 

public 

SP 

+ 

+/- 

n.a. 

444.000 



CBI 


Philexport 

Cebu 


EMM 

+ 

+/- 

n.a. 

495,000 

+7% 


CBI 




Ml 

+ 

+ 

n.a. 




ext. 

Vietnam 

HCACS 

public 

SP 

+ 

-/+ 

n.a. 

n.a. 

n.a. 


ext. 

Indonesia 

NAFED/ 

DGNED/ 

IETC 

public 

SP/EMM 

■■ 



808,000 

- 39% 


ext. 

Indonesia 

IETC 

public 

SP/EMM 

+ 

+ 

+/- 




ext. 

India 

TEDO 

private 

EMM 

*/- 

-/+ 

-/+ 

426,000 

- 22 % 

network of 
trained 
local experts; 
'Value M agics’ 
publication; 
pilotSMEs 

EMM 

knowledge 

ext. 


TEDO 


SP 

- 

■** 

- 

301,000 


7 coördinators 
trained 

ext. 


TEDO 


Ml 

+ 

-/+ 

-/+ 

299,000 


20 ready-to- 
use market 
reports, 

MBA syllabus 

ext. 

Peru 

PROM- 

PERU 

public 

SP 

+ 

+ 

-/+ 

478,000 

+8.7% 


ext. 

Peru 

ADEX 

private 

SP/EMM 

+ 


+/- 

842,000 

- 4.3% 


ext. 

Bolivia 

CANEB 

private 

SP/MD 

+ 

+/- 

-/* 

966,000 

+ 10 % 


ext. 

Bolivia 

IBCE 

private 

SP/Ml 

+ 

+ 

+ 

814,000 

- 7.5% 


ext. 

Ecuador 

Corpei 

public 

Ml 

+ 

- 

- 

n.a. 

+3.5% 



Legend: + = good; — = bad; n.a.= not available. 
Source: CBI regularevaluations. 
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The First column indicates the information source, either an evaluation / peer review 
conducted by CBI experts, or an evaluation conducted by an independent external 
organisation. The second and third columns mention the beneficiary country and 
beneficiary name. The ‘status’ column indicates whether the BSO is a publicly or privately 
administrated organisation. The modules delivered to the BSOs are mentioned in the 
following column.' 67 This is followed by an assessment of the relevance, effectiveness, and 
efficiency of the respective modules. The expenses and the overspending and 
underspending, respectively, is mentioned in the following columns (in some cases no 
details per module were available, which is why integrated expenditures per programme are 
listed instead). The outputs, to the extern that they are available, and not always in full, have 
been summed up in the final column. 

Modules that either severely underspent or overspent were nevertheless implemented. Both 

can point to either poor planning or inefficiënt programme delivery. The latter is believed 

to be the most likely case. Also, an examination of the expenditures against the delivered 

outputs suggests that programme delivery could have been more efficiënt. In the case of 

IPTC the average costs of a trained local consultant were at least EUR 11,000, which is rather 

expensive for the type of training delivered. Also the seven coördinators trained in the TEDO 

strategie planning module were trained for an average cost of EUR 43,000, which is not a 

sign of efficiënt programme delivery. | ns 

An unknown number of local experts were trained in the TEDO EMM module, which costed 
EUR 426,000, and a publication written by the CBI expert and local consultant' 68 was 
financed from the budget. Furthermore, some pilot SMEs have been equipped with EU 
export marketing and management knowledge. The market information module, costing 
EUR 299,000, has delivered twenty ready-to-use market reports, an MBA syllabus, and access 
to EU market information from a CBI database. The cost for this seems high, ifyou consider 
that these are no tailor-made products but outputs of other CBI products. The actual 
expenditures for the MI module delivery at Philexport went over budget by 12%. This was 
caused by additional efforts, i.e. two additional missions for two external experts, to make 
the achieved results more sustainable. 


’ 67 The exact results achieved by a BSOD programme will depend on the particular organisational needs of 
the BSO, as well as the country and sector needs. These needs are assessed during the programme’s 
preparatory phase. In this stage, one or more of the four available BSOD modules are identified that 
would best address these particular needs. These modules should be seen as focus areas of the BSOD 
intervention. They are adapted to the needs of each particular BSO, and are not Standard ‘one-size-fïts- 
all' Solutions. A module can span a period of three to fïve years. 

168 lt is not dear if the publication (titled ‘Value Magics') should be considered as part of the external 
experts’ assignment. This is but one example of a situation in which a CBI expert, besides having a 
contract with CBI, is also personally involved in business activities related to the CBI assignment. 
Another example is the expert that starts to become the EU importer fora company, which was 
selected into the ECP programme by the expert him/herself. This seems to be undesirable, as it triggers 
potentially adverse effects. 
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As for the modules that underspent, the evaluator concluded that these interventions could 
have been implemented in a more efficiënt manner if the difference between the budget 
and the expenditure had been used to train more experts and facilitate a close and strong 
partnership agreement between locally supported Indian beneficiaries. One year after the 
completion of the programme, at the time of the evaluation, this partnership was still 
unofficial. 

NAFED also underspent, to the tune of 40%. It suggests that the programme might have had 
more impact if it could have used the remaining budget to develop more activities for 
people from NAFED and IETC, consult and coach more Indonesian SMEs, or even support 
one or two potential SMEs to export to EU markets. Unfortunately, the evaluation team was 
unable to obtain any information on the cancelled activities. No document is available, and 
according to in-depth interviews, no activity was reported as cancelled. Due to the 
unavailability of an expenditure breakdown for either the modules or the BSOs, it was not 
possible to arrivé at any kind of specific conclusion on the efficiency of the programme. 

The support of the (privately organised) IBCE, on the other hand, is an example of an 
effective and efficiently run intervention, in part thanks to the commitment and ownership 
of the programme by the beneficiary. Targets were achieved despite spending being 7.5% 
under budget. This is an example of the efficiënt use of resources by both IBCE and CBI, who 
prioritised interventions that could have greater impact, and when possible hired a local 
consultant that worked at IBCE for much less than a European consultant would have cost. 

There was a solid balance between strategie activities for capacity development and 
technical assistance for the market information module. Implementation took one year 
longer than originally programmed. Additional time was necessaiy, mostly to complete the 
programme and to fme-tune some of the results from completed activities. 

Due to changes in the accounting system of CBI, the evaluators did not have access to the 
information that would have allowed a detailed comparison of the budgeted amounts and 
expenditures by module and by activity in each BSO. However, in general terms, and based 
on the achieved effects, it appears that the amounts spent in CANEB and ADEX were the 
largest, but at the same time, these BSOs were the ones with more mixed results. 
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6.5 Conclusions 


• The increase of trade promotion organisations means the relevance and additionality of 
interventions are now at stake, because good coordination between service providers is 
often lacking, and this has led to a suboptimal division of tasks. The relevance of 
technical assistance to BSOs depends very much on the beneficiaries’ mandate. Their 
fmancing is often not secure and their choice of personnel is often politically 
motivated. Generally, the BSOD interventions reviewed were considered to be relevant. 

• The effects of the technical assistance provided to BSOs is hard to measure accurately, 
given that the quality standards set by MFA and CBI are too general. Anecdotal and 
observable evidence produces a mixed picture of the BSOD programmes’ effectiveness. 
Evaluations indicate that more focused sector-oriented BSOs, which can play a key role 
in facilitating and mobilising a network of local experts to promote export, seem the 
most likely candidates to have an impact. The assumed effectiveness is not underpinned 
by strong facts or metrics. 

• The use of newly acquired competences is less obvious at the BSO level, since CBI 
interventions are predominantly an investment in individual development rather than 
in organisational development. 

• There is no evidence that the assumed improved service delivery by supported BSOs to 
dient exporters also leads to increased export turnovers for these clients. Since most 
anecdotal and observable evidence does not point towards the claimed effect, stronger 
facts or metrics that would underpin the opposite still need to be presented. 

• Due to a lack of reliable and valid programme monitoring data, there is still substantial 
empirical uncertainty regarding efficiency. Based on what we know about the 
programme costs and outputs, we conclude that the implementation of the BSOD 
programme generally lacked efficiency, both from an institutional and programmatic 
perspective. 

• From a programme management perspective, we conclude that the lack of data severely 
hampers attempts to monitor and steer programmes in the desired direction. 


I117 


Annexes 



Annexes 


Annexi AboutlOB 

Objectives 

The Policy and Operations Evaluation Department (IOB) aims to contribute to knowledge 
of the implementation and impact of Dutch foreign policy. IOB meets the need for 
independent evaluation of policy and operations in all the policy fields of the Homogenous 
Budget for International Cooperation (HGIS). IOB also advises on the planning and 
implementation of evaluations falling under the responsibility of the policy departments 
of the Ministry of Foreign Affairs of the Netherlands and its embassies. 

IOB’s evaluations enable the Minister of Foreign Affairs and the Minister for Foreign Trade 
and Development Cooperation to give account to Parliament for their policies and for 
resources spent. In addition, the evaluations aim to contribute to leaming by formulating 
lessons and options for policy improvements that can be incorporated into the ministry's 
policy cycle. Insight into the outcomes of implemented policies allows policymakers to 
devise new policy interventions that are both more effective and better targeted. 

Organisation and quality assurance 

IOB has a staff of experienced evaluators and its own budget. When carrying out 
evaluations, IOB calls on specialist knowledge from external experts with knowledge of the 
topic under investigation. By way of quality control, IOB appoints an external reference 
group for each evaluation, which includes not only external experts, but also relevant 
policy-makers from the ministry and other experts. Moreover, for each evaluation IOB 
appoints several of its own evaluators to act as peer reviewers. IOB's Evaluation policy and 
guidelines for evaluation are available on the website www.iob-evaluatie.nl, hard copies can be 
requested through the IOB secretariat. 

Evaluation programming 

IOB consults with the policy departments to draw up a ministry-wide evaluation 
programme. This rolling multi-annual programme is adjusted annually and included in the 
Explanatory Memorandum to the ministry’s budget. IOB bears ftnal responsibility for the 
programming of evaluations in development cooperation and advises on the programming 
of foreign policy evaluations. The themes selected for evaluation respond to requests from 
the ministry and Parliament and/or are considered relevant to society. IOB actively 
coordinates its evaluation programming with that of other donors and development 
organisations. 

Approach and methodology 

IOB aspires to relevance, high quality and methodological innovation. Whenever possible, 
the research applies both quantitative and qualitative methods Ieading to robust impact 
evaluations. IOB also undertakes systematic reviews based on empirical results relating to 
priority policy areas. IOB has extended its partnerships with evaluation departments in 
other countries, for instance through joint evaluations and evaluative knowledge 
exchanges, undertaken under the auspices of the OECD-Development Assistance 
Committee NetWork on Development Evaluation. 
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Annex 2 Organisational chartof CBI 

Organisational Chart 


Business Operations 



Managing Director | 


1 


Research & Development 

- Marketintelligence 

- Marketing & communication 

- Sector- and Theme expertise 

- Learning 

- Strategy 

- Monitoring & Evaluation 





Regional team 

Regional team 

Regional team 

- Asia 

- Africa 

- Latin America 

(including 

(including 


Eastern Europe) 

Middle East) 





international network of external experts 


120 | 


Source: CBI ( 20130 : 60 ). 





























Annexes 


Annex3 Documentation reviewed 


AIV (2012). Ongelijke werelden, armoede, groei, ongelijkheid, en de rol van internationale samenwerking. 
Dutch Advisory Committee on International Development Cooperation. The 
Hague, 2012. 

Bank, S. andTersteeg, J. (2009). NetherlandsTrustFundwith International TradeCentre. Report of the 
extemalevaluation 2005-2008. Rotterdam: CBI. January 2009. 

Cadot, et al. (2011). Are Export Support Programs Effective?EvidencefromTunisia. Washington, 

D.C.: World Bank. 

Cadot, et al. (2011). Impact Evaluation of Trade Interventions. Washington, D.C.: World Bank. 

CBI (2010). CBI Jaarverslag 09. Rotterdam: CBI. March 2010. 

CBI (2011). CBI Jaarverslag 10. Rotterdam: CBI. May 2011. 

CBI (2012a). CBI Annual Report 2011. The Hague: CBI, Ministry ofForeign Affairs of the 
Netherlands. May 2012. 

CBI (2012b). CBI Program Management Manual. Developing Integrated Export Value Chain Programmes. 

The Hague: CBI, Ministry of Foreign Affairs of the Netherlands. July 2012. 

CBI (2012c). Business Case CBI 2013-2017. Shaping the Future. The Hague: CBI, Ministry of Foreign 
Affairs of the Netherlands. October 2012. 

CBI (2013a). CBI Annual Report 2012. The Hague: CBI, Ministiy ofForeign Affairs of the 

Netherlands. May 2013. http://www.cbi.eu/sites/default/files/fUes/CBI_annual_report2012.pdf 
CBI (2013b). Centre for the Promotion oflmportsfrom Developing Countries. The Hague: CBI, Ministry 
ofForeign Affairs of the Netherlands. http://www.cbi.eu/sites/default/files/publications/ 
Corporatebrochure-English.pdf 

CBI (2013c). CBI the Expert in Export. The Hague: CBI, Ministry ofForeign Affairs of the 
Netherlands. 

http://www.cbi.eu/sites/defauIt/files/publications/CBI_Expen_in_Export_VL2.pdf 
Ecorys (2011). Evaluation of ECP Programmes ITO, BPO, Timber and timber-relatedproducts. Draft final 
report. Rotterdam: Ecorys, October 2011. 

EFECE (2013). External Evaluation of the BSOD Programme in Latin America (2006-2011). Final Report. 
La Paz: EFECE &Asociadoss.r.l., May 2013. 

EIM (2007). CBI Market Information Programme. Final Evaluation Report. Zoetermeer: EIM, 

May 2007. 

EIM (2012a). CBI Export Coaching Programmes (ECP). Evaluation Report 5 sector programmes 
Part I - Main Report. Zoetermeer: EIM, March 2012. 

EIM (2012b). CBI Export Coaching Programmes (ECP). Evaluation Report 5 sector programmes. 

Part II - Annexes. Zoetermeer: EIM, March 2012. 

EIM (2012c). CBI Export Coaching Programmes (ECP). Evaluation ofthree ECP programmes. Final 
Report. Zoetermeer: EIM, December 2012. 

iLumtics (2013). BSOD Programme Evaluation 2012-2013 TEDO (India), DGNED/IETC (Indonesia) and 
FICACS (Vietnam). iLumtics Vietnam, March 2013. 

Evaluatiegroep (2008). Evaluation of the CBI Commercial Attaché Training Programme. Utrecht: 
Evaluatiegroep Overheid en Bedrijf, April 2008. 


|121 


Aided trade 


122 | 


IOB (1999). Hulp door handel. Evaluatie van het Centrum tot bevordering van de Import uit 

Ontwikkelingslanden (CBI), 1990-1996. IOB Evaluation 281. The Hague: Inspectie 
Ontwikkelingssamenwerking en Beleidsevaluatie. October 1999. 

ISB (2009). Verkorte organisatiedoorlichting Centrum tot Bevordering van de Import uit 
Ontwikkelingslanden (CBI). MFA/ISB. 

Lederman et. al. (2010) Export Promotion Agencies: What Works and What Doesn’t. Washington, 

D.C.: World Bank. 

MDF (2007a). Evaluation CBI Market Access Requirement seminars and workshops - 
MASTER and MAR 2003-2006. 

MDF (2007b). On thePath toFame. Evaluation CBI Training Course Facilitators and Advisors for 
Marketing and Export Promotion (FAME) 2003-2007. Rotterdam: MDF, November 
2007. 

Novatech (2011). Corpei’s Market Information System: Evaluation Report. CBI BSOD 
Programme Corpei 2006-2009. Quito: Grupo Novatech, April 2011. 

Panteia (2013). Towards betterMIproducts. Resultsofthesurveyon the quality of CBI’s MI products. 
Zoetermeer: Panteia, Januaiy 2013. 

ProFound (2007). Final Evaluation of CBI’s Human Resource Development Programme Workshops 
2000-2006. Utrecht: Profound, June2007. 

Triodos Facet (2009). Review and Summary of CBI Evaluations 2003-2009. Zeist: Triodos Facet, 
October 2009. 

WEF (2012). The Global Competitiveness Report of the World Economie Forum. 

Literature review 

Alvarez, R., Crespi, G. and Volpe Martincus, C, (2012). Impact Evaluation in a Multiple Program 
World. Inter-American Development Bank. 

Bernard, A.B. and lensen, J.B. (2004). Why some firms export. In: RevEcon Stat, 86: 561-569. 

Cadot, O., Carrère, C. and Strauss-Kahn, V. (2011). Export diversification: What’s behind the 
hump? RevEconStat, 93: 590-605. 

Cadot, O., Fernandes, A.M., Gourdon, J. and Mattoo, A., (2012). Are the benefits of export support 
durable? Evidencefrom Tunisia. 

Creusen, H. and Lejour, A. (2011). Uncertainty and the export decisions of Dutch firms. FIW. 

Cruz, M. (2014). Do Export Promotion Agencies Promote New Exporters? Inter-American 
Development Bank. 

Czinkota, M.R. (2011). The Rationale for Export Promotion, http://michaelczinkota.com/ 

wp-content/uploads/2011/ii/The-ra tionale-for-export-promotion-i.pdf (accessed 
6/20/14). 

Dollar, D. and Kraay, A. (2002). Growth is good for the poor. In: lournal ofEcon Growth, 7: 
195-225. 

Dollar, D. and Kraay, A. (2004). Trade, growth, and poverty. In: Economie Journal, 114: F22-F49. 

Frankel, J.A. and Romer, D. (1999). Does trade cause growth? In: Am Econ Rev, 379-399. 

Freund, C. and Bolaky, B. (2008) Trade, regulations, and income. In: / DevEcon, 87: 309-321. 
doi: io.ioi6/j.jdeveco.2007.n.oo3. 

Gil, S., Llorca, R. and Martinez Serrano, J.A. (2008) Measuring the impact of regional export 
promotion: The Spanish case. In: Pap. Reg. Sci. 87,139-146. doi: 

10.1111/j.1435-5957.2007.00155.x. 


Annexes 


Giles, J.A. and Williams, C.L. (2000). Export-led growth: A survey of the empirical literature 
and some non-causality results. Parti. In: 1 ntTradeEconDev, 9: 261-337. 

Görg, H., Henry, M. and Strobl, E. (2008). Grant support and exporting activity. In: RevEcon 
Stat, 90:168-174. 

Harrison, A. and Rodriguez-Clare, A. (2009). Trade, foreign investment, andindustrialpolicy for 
developing countries. National Bureau of Economie Research. 

Hausmann, R., Hwang, J. and Rodrik, D. (2007). Whatyou export matters. In: ƒ Econ Growth, 
12:1-25. 

Hesse, H. (2006). Export diversification and economie growth. Washington, D.C.: World Bank. 

Hogan, R, Keesing, D.B., Singer, A. and Mundial, B. (1991). The rok of support services in 

expanding manufactured exports in developing countries. Washington, D.C.: World Bank. 

Lall, S. (1997). Selective policies for export promotion: lessonsfrom theAsian Tigers. UNU World 
Institute for Development Economics Research. 

Lall, S. (2004). Selective industrial and trade policies in developing countries: theoretical 
and empirical issues. In: Polit. Trade Ind. PolicyAfr. EorcedConsens, 4-14. 

Lederman, D., Maloney, W.F., et al. (2007). Trade structureand growth. In: NatResourCurse 
Destiny, 15-39. 

Lederman, D., Olarreaga, M. and Payton, L. (2010). Export promotion agencies: Do they 
work? In: I Dev Econ, 91: 257-265. 

Martincus, C.V. and Carballo, J. (2010a). Export promotion: Bundled services work betten 
In: WorldEcon, 33:1718-1756. doi: 10.1111/j.1467-9701.2010.01296.x. 

Martincus, C.V. and Carballo, J. (2010b). Beyond the average effects: The distributional 

impacts of export promotion programs in developing countries. In: ] DevEcon, 92: 
201-214. doi: io.ioi6/j.jdeveco.2oog.o2.oo7. 

Martincus, C.V., Carballo, J. and Garcia, P.M. (2012). Public programmes to promote firms' 
exports in developing countries: Are there heterogeneous effects by size 
categories? In: AppEcon, 44: 471-491. doi:io.1080/00036846.2010.508731. 

Ravallion, M. (2001). Growth, inequality and poverty: Looking beyond averages. In: World 
Dev, 29:1803-1815. 

Rodrik, R. (1995). Chapter 45, trade and industrial policy reform. In: [ere Behrman and T.N. 
Srinivasan (eds.), Handbook of Development Economics. Elsevier, 2925-2982. 

Romer, D. (1993). Openness and inflation: Theoiy and evidence. In: QJEcon, 108: 869-903. 

Romer, P. (1994) New goods, old theoiy, and the welfare costs of trade restrictions. / Dev Econ, 
43: 5 - 38 . 

Sachs, J.D., Warner, A., Aslund, A. and Fischer, S., (1995). Economie reform and the process 
ofglobal integration. Brook Pap Econ Act, 1-118. 

Schminke, A.D. and Van Biesebroeck, J. (2013). Evaluation of export promotion policies in Belgium. 

SIDA(20i3). What works for market development: Areview of the evidence. 

Volpe Martincus, C, and Carballo, J. (2008). Is export promotion effective in developing 
countries? Firm-level evidence on the intensive and the extensive margins of 
exports. In: ƒ Int Econ, 76: 89-106. 

Volpe Martincus, C. and Carballo, J. (2012). Export promotion activities in developing 

countries: What kind of trade do they promote? In: I Int Trade Econ Dev, 21: 539-578. 

Wacziarg, R. and Welch, K.H. (2008). Trade liberalization and growth: New evidence. 

In: World Bank Econ Rev, 22:187-231. 


Aided trade 


Winters, L.A. (2004). Trade liberalisation and economie performance: An overview. 

In: Econ), 114: F4-F21. 

Winters, L.A. and Martuscelli, A. (2014). Trade liberalization and poverty: What have we 
learned in a decade? In: AnnuRevResourEcon, 6: 493-512. 

Winters, L.A. and Masters, A. (2013). Openness and growth: Still an open question? 

In: ]. Int. Dev. 25,1061-1070. doi: 10.1002Zjid.2973 
Winters, L.A., McCulloch, N. and McKay, A. (2004). Trade liberalization and poverty: 
The evidence so far. In: ] Econ Ut, 72-115. 


I 124 I 


Annexes 


Annex 4 Interviewees 


Name 

Function 



Mr Ton Lansink 

Former Director CBI (2002-2008) 

Mr Hans Klunder 

Former Director CBI (2008-2014) 

Mr Dick de Man 

Deputy Managing Director CBI (CBI since 1995) 

Ms Henrique Postma-Hazelaar 

Programme manager 1040 

Ms Melanie Haga 

Programme manager 1039/38/34 

Mr Cor Dieleman 

Programme manager 1028/29/36 

Mr Wim van Heumen 

Programme manager 1031/43 

Mr Jan Oude Eferink 

Expert 1028 

Mr Arjan Hoek 

Expert 1030 

Mr Lazlo Klucs 

Expert 1030/1032 

Ms Aileen Brindle 

Expert 1040 

Mr Jan Ramakers 

Expert 1043 

Mr Bouke van de Pol 

Managerial controller CBI 




Mr Ad Lansu 

Senior account advisor (DDE 169 / CU 170 ) 

Mr René Rotteveel 

Controller DDE/CU 

Mr Sjoerd Smit 

former account advisor CBI at DDE 

Ms Henny de Vries 

Account advisor CBI at DDE 

Mr Menno Zijlstra 

Accountant The Central Government Audit 

Service 




Mr Robert Dijksterhuis 

Director at the Netherlands Enterprise Agency 
(RVO) 


169 Sustainable Economie Development Department. 
' 7 ° Control Unit. 
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Annex 5 Evaluations by CBI 


Recent CBI evaluations 

Year 

Interventions 

evaluated 

Some details of the evaluations 

2009 

4 BSODs 

Support programmes in Vietnam, East Africa, Philippines and Jordan 

2011 

5 ECPs 

ECPs in the areas of mobile equipment parts, engineering products, 
subcontracting, medical devices and laboratory equipment, and 
pharmaceutical products, which received extremely high satisfaction 
ratings from participants 


3 ECPs 

IT Outsourcing, Business Processes Outsourcing, Timberand 
timber-related products 


2 BSODs 

2006-2009 assistance to Corpei Ecuador; assistance to IBCE Bolivia 

2012 

3 ECPs 

Three ECPs fïnalised towards the end of 2012 in the areas of 
outerwear (Macedonia), food ingredients, and home decoration, 
accessories b gifts 

2013 

2 BSOD 

programmes 

Asia (India, Indonesia and Vietnam) and Latin America (Bolivia and 

Peru) 


So urce: CBI annual reports and various evaluation reports. 
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Annex 6 Working Instruction Export/ 
Business Auditfor ECPs 



Source: CBI, Working Instruction Export/Business Auditfor ECPs. 

1127 

The figure illustrates the modus operandi of an ECP. Every company that receives export 
coaching from CBI is audited on various (clusters of) aspects. The working instruction 
provides insight into the relation between the audit and the various modules of an ECP. 171 
The audit scores determine which intervention pathway to offer the company. The working 
instruction underlines that the intervention does not focus solely on decreasing 
Information asymmetry, but also reaches out to business development in general. 

Moreover, the figure also shows that CBI targets companies with a broader range of needs, 
since beneficiaries' scores on critical clusters lie in the range of grades 1-4. 172 

According to advocates of a more strict interpretation of CBI's mission, the audit was only 
meant to be a selection tooi. Instead, the audit has become a diagnostic tooi based on 
which action plans are being developed in order to address these companies’ weaknesses. 173 
Advocates of this view argue that if these weaknesses should be addressed in order to 
include a company in the programme, CBI should leave it to local companies to provide 
such support. According to those that believe CBI's main mission is to reduce information 
asymmetry, CBI should have been more selective when choosing its beneficiaries, only 
taking companies on board that were on the brink of exporting to the EU, since CBI is 
explicitly not meant to be a development organisation. Consequently, this line of reasoning 
also ex ante denied the potential effectiveness of the CBI programme in least-developed 
countries. Nevertheless, advocates of this Vision were unsuccessful. 

171 These modules are export audit and action plan, business development, export capacity building, 
certification, and market entry regional/EU. 

172 A similar argument about a broad strategy approach can be upheld for interventions within the BSOD 
programme, in which a wider variety of organisations (including, for example, government export 
departments, chambers of commerce, or trade/sector associations) have been supported. 

173 One interviewee specifically typifies this development as ‘mission creep'. 
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Annex7 Reconstructed intervention logic of 

ECP and BSOD programmes 
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Annex 8 Result chain private sector 

development policies 2005-2012 
(reconstruction by IOB) 


Economie growth £7 reduced poverty 

Higher 

investments 

(national 

ana hui) 

Stronger 

companies 

Betterjobs 

Productivity 

increase 

Better 

company 

results 

Less imuniidi 

economy 

Betteraccess of 
infrastructure. 

Reduced transaction 
costs. 

Betteracces 
(innovative) 
fïnancial services. 

Improved production, 
production processes 
and marketing. 

Innovative production. 

Improved 

entrepreneurship. 

Stronger social 

dialogue. 

Law and regulatory 

reforms (indicator: 

Doing business scores) 

Roads, bridges, harbours, 
electrification, hospita Is, 
water b sanitation, 
irrigation, telecom. 

More widespread and 
improved fïnancial service 
delivery. 

New fïnancial products 
Financial sector reforms. 

Reports on products and 
production processes. 
Market chain partnerships. 

Joint ventures. 

Company advises. 

Trained employers 
and employees. 

Better organised producer 
organisations. 

Reports, proposals, 
seminars, aimed at law 
and regulatory reform. 

Co-financinglocal 

infrastructure. 

Delivery of Capital 
goods and technology. 

Financingand 
support of fïnancial 
intermediaries. 
Development of 
innovative fïnancial 
services. 

Advisory services 
fïnancial law and 
regulations. 

Advisory services on 
products/services and 
production methods. 
Developing partner¬ 
ships forsustainable 
market chains. 

Co-fïnancing of joint 
ventures. 

Analysing business 
processes. 

Financing vocational 
education. 

Supporting producer 
organisations. 

Analysis of business 
climate constraints. 
Technical assistance 
to(local)govern- 
ments. 

ORET, 

FMO-IDF, 

PIDG.ORIO 

FMO-MASSIF, 

NI PR GTLR 
FMO-CD, TCX 

CBI, IDH, 

Solidaridad, 

IFDC 

PSOM/PSI, 

PUM, 

Agriterra, 
Woord en 

Daad 

Bilateral 
programmes 
in Bolivia, 
Rwanda, 
Zambia, etc. 

1 FC Africa 

Infrastructure 
(EUR 1.171 bln.) 

Financial sector 
(EUR 736 bln.) 

Market 
development 
(EUR 550 bln.) 

Knowledge 
transfer 
(EUR 615 bln.) 

Law and 
regulation 
(EUR 60 mln.) 
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Annex 9 Brief description CBI evaluation 
framework 


CBI evaluates its programmes, the results ofwhich are judged according to the defined 
result indicators of the programme interventions. The relevance and quality of these 
indicators are not at stake during these regular evaluations. CBI (result/effect/impact) 
programme indicators have been formulated to serve two goals: (1) project monitoring 
(content), and (2) project administrative purposes (ex ante price-setting mechanism). 

The CBI Programme Management Manual lists some key performance indicators (KPIs) per 
intervention area at both the output and outcome levels. 174 These KPIs are presented in the 
table below. However, this set of indicators was defined in 2012, whereas the period under 
evaluation is 2005-2012. Hence, it would not be fair to assess the CBI interventions on the 
basis of these indicators if they were not used in the precedingyears as welk 
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Table A Key performance indicators per intervention area 
Intervention Area KPIs Measured by 


ECP (Exporters) 


BSOD (Chain 
supporters) 


Policy (Chain 
influencers) 

Importers 


Source: CBI ( 2012 b: 22 ). 


# of competent exporters 

# of exporters that have establis- 
hed number of relevant business 
contacts with EU/regional 
importers 

increase in exports (outcome 
indicator) 

# of clients of BSOs that use related 
service (uptake) 

increase in exports (outcome 
indicator) 

# of improved trade policies 
increase in exports (outcome 
indicator), if relevant 175 

# of ongoing partnerships with 
BSOs in target export markets 
increase in exports (outcome 
indicator), if relevant 


internal audits & external dient 
satisfaction surveys 
external exporters survey 


internal reporting/assessments 
external cliënt satisfaction surveys 


internal assessment 
documentation or exporter surveys 

external exporters survey 
external exporters survey 


' n CBI ( 2012 b: Table 4 and Annex 8 ). 

175 Policy as well as importer projects may not necessarily contribute directly to outcome (i.e. export), but 
only support the efforts of other (e.g. ECP) projects. In these cases, no increase in exports needs to be 
reported. 
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Actually, a different, though partially overlapping set of indicators was defined earlier.' 76 
This set includes indicators on the so-called ‘result’, ‘effect’ and ‘impact’ levels, which were 
formulated with the introduction in 2006 of the ex ante price system. 177 Table B gives an 
overview of the ‘quality indicators’ that were formulated for ECP and BSOD programmes, as 
well as indicators for market information and training. Norms have been specified for these 
indicators. 


Table B < 

Quality indicators per intervention area and level 

Intervention 

area 

‘Result’ 

(copacity transfer) 

‘Effect’ 

(use of capacity) 

‘Impact’ 

(effect of use of capacity) 

ECP 

(Exporters) 

• Sufficiënt score (i.e. at 
least 2 on a scale of 4) 
for the clusters of the 

CBI export audit. 

• Export marketing plan 
that reflects knowledge 
of EU/EFTA markets 
and export marketing. 

• Sufficiënt# of relevant 

business contacts in 

the EU market and 

demonstrated skills 
and experience in 
product marketing. 

• # of businesses 

with demon¬ 
strated exports 
to the EU/EFTA. 

• Total additional exports to EU/ 
EFTA one year after completion 
of the CBI intervention. 

• Total additional exports to 
non-EU/EFTAfrom the start of 
the CBI intervention until one 
year after its completion. 

• Additional number of jobs 
measured from the start of the 
CBI intervention until one year 
after its completion. 

BSOD 

(Chain 

supporters) 

• Certification of transfer 
of competences. 

• Use of 
competences. 

• Users/clients of the services 
provided by BSOs judge that 
they receive a better service 
provision than they received 
prior to the support given by 

CBI. 

Market 

information 

• # of received units of 

market information. 

• Received market 

information 
actually read. 

• Usefulness of received market 

information. 

• Enhanced knowledge of EU 
export market. 

Training 

• More insight and 
knowledge of 
exporting. 

• Use of insight 
and knowledge. 

• Additional exports or more 
effective service provision. 


Source: CBI ( 2007 ), Memorandum of 16 November 2007 ; CBI ( 2008 ) Memorandum of 31 March 2 oo 8 ;seeatso EIM 
( 2012 c). 


’ 76 This set is documented in a Memorandum from CBI to DGIS dated 16 November 2007 . 

177 Memorandum from CBI to DGIS dated 16 November 2007 ; Memorandum from CBI to DGIS dated 
31 March 2008 . The three levels appear to be equivalent with the output, outcome and impact levels in 
a results chain. 
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Annexio Enablingtrade dimensions- 
country scores 



The amount spent on ECPs is represented by the size of the blocks. The colour of the blocks 
represents the assessment on the ETI dimension ‘market access’. Countries marked in green 
have good prospects for accessing markets, while the prospects for those marked in red are 
generally judged to be unfavourable. The orange-coloured countries hover in between. Each 
block further indicates the country name, followed by the CBI country group indicator (exit, 
PSD, transition, and focus countries), the country income group (lower-income country, 
lower-middle-income country, upper-middle-income country,), and the assessment on the 
ETI dimensions ‘infrastructure’ and ‘operational environment’. 
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Annex n Employment effects reviewed 


CBI testimonials 

Reassessment 

Programme 

Description 

Employment 

increase 

Competent 

exporters 

Increase per 

competent 

exporter 

Employment 

Increase 

1028 

Mobile equipment parts 

1,270 

30 

42 

660 

1029 

Engineering products 

861 

32 

27 

640 

1030 

Medical devices and 
laboratory equipment 

343 

37 

9 

343 

1031 

ITO 

836 

38 

22 

570 

1032 

BPO 

-100 

10 

-10 

-100 

1034 

Pipes and process equipment 

4,051 

47 

86 

2,714 

1035 

Outerwear 

-1,533 

21 

-73 

-1,533 

1036 

Sub-contracting castings and 
forgings 

593 

38 

16 

593 

1037 

Food ingredients 

0 

29 

0 

0 

1038 

Personal and protective 
equipment 

373 

25 

15 

n.a. 

1039 

Office and school supplies 

349 

47 

7 

-159 

1040 

Home decoration accessories 
and gifts 

2,303 

44 

52 

14 

1043 

Pharmaceutical products 

89 

27 

3 

89 

Total 


9,435 

425 

22 

3,831 


Source: CBI regutar evaluations, IOB survey. 


According to programme management information obtained from CBI, which has been 
collated in this annex, there have been substantial job increases in ‘pipes and process 
equipment’, 'home decoration accessories and gifts' and 'mobile equipment parts’ 178 , while 
the 'outerwear’ programme in Macedonia has suffered major job losses.™ Another regular 
ECP evaluation indicated that the total number of jobs attributed by 31 respondents was 471 
(an average of tsperenterprise), but a disclaimer stated that more than 110 additional jobs 
were reported by one company, so the average number of jobs per company was 
exaggerrated. 180 

’ 7S This number seems to be rather high bearing in mind the discussion on the trade-off between market 
destinations, which suggests there has not been an increase in funds. 

’ 75 The evaluation of ‘business process outsourcing’ ( 1032 ) concluded that at the end of the programme in 
2009 , all companies had to reduce the size of their staff as a result of the economie crisis. By contrast, 
the evaluation of the ITO programme ( 1031 ) suggested that the programme has had a positive and quite 
significant impact. 

’ So EIM 2011 / 2012 . 
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Annexi2 Benchmarkmethodology 


The following set of criteria has been used to identify and assess the appropriateness of 
potential benchmark organisations: 

• Organisation promoted imports from developing countries between 2005 and 2012 
(required); 

• Operational focus of the import promotion programme is on individual businesses in 
developing countries (required); 

• Import promotion programme aims to increase exports (end or intermediate goal) (required); 

• Organisation has data available on (the development of) export turnover of supported 
businesses in developing countries (required); 

• Organisation has similar import promotion strategy to CBI, including technical 
assistance/ coaching of individual businesses, dissemination of market intelligence, etc. 
(preferred); and 

• Import promotion programme has similar sector and geographical focus (preferred). 181 

These criteria were applied to all member organisations of the Trade Related Instruments 
Connected (TRIC) network, known as the TRIC partners, in accordance with CBI advice. 182 

The activities of these organisations were screened on the basis of their websites and annual 
reports. Some organisations were contacted with requests for additional information. Our 
conclusion was that all IPOs, except VIRKE, have promoting imports from or private sector 
development in developing countries as one of their goals, but the strategies that most of 
these organisations employ to reach these goals are different from CBI’s. 

The Swiss Import Promotion Programme matches the criteria and shows the greatest 
resemblance to CBI’s export coaching programme. The programme works in similar sectors 
as CBI, i.e. jewellery, agricultural and forestry products, flsheries, vehicles, and electronics 
(seeBoxgbelow). 

A second comparable organisation, the Trade for Development Centre, a programme of the 
Belgian Development Agency, meets the three required criteria but was not able to provide 
information on the outcome of their activities in terms of increased exports. The sector 
focus is also slightly different from CBI, with a major focus on (small) agricultural 
businesses and no industrial products at all. Even so, TDC is included in the benchmark for 
the purpose of comparing the cost structures of the IPOs and their approaches to export 
promotion. 


181 lf a criterion is ‘required’, this means organisations have to meet it to be able to serve as a benchmark 
for CBI. If a criterion is ‘preferred’, a higher score on this criteria means higher eligibility. 

181 Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), Import Promotion Desk (IPD), Swiss 
Import Promotion Programme (SIPPO), Finnpartnership, The Enterprise Federation of Norway (VIRKE), 
Trade for Development Centre (TDC) and Chamber Trade Sweden. 


Annexes 


Box 9 Swiss import promotion programme 

The Swiss Import Promotion Programme (SIPPO) is an instrument of economie 
development cooperation, under the responsibility of the State Secretariat for 
Economie Affairs (SECO) of Switzerland. It was founded in 1985 to promote SMEs' 
access to Swiss and EU markets. SIPPO operated independently in its fïrst decade, 
but was integrated into Switzerland Global Enterprise in 2007 and 2008. This is also 
the dedicated agency that promotes Swiss exports abroad and tries to attract 
inward foreign investments. 

SIPPO’s operational focus during most of the evaluation period was on bringing 
individual businesses to the European market, via trade fairs and coaching. The 
target group consisted of small and medium-sized businesses on the verge of 
export-readiness. SIPPO’s products - training, matchmaking, and market informa- 
tion - closely resembled CBI’s. However, as recommended in the 2011 evaluation 
report, SIPPO has started to move away from increasing individual businesses’ 
competitiveness towards building the capacity to support local business and 
promoting market development. This change is being implemented gradually, as 
shown by the 2012-2015 Performance Contract. Two new approaches are added to 
the portfolio in this new contract: ‘systemic market development’ and ‘country 
pavilion’. These approaches, which were fïrst developed in pilot projects, are now 
being incorporated into SIPPO’s modus operandi. Even so, a considerable share of 
the budget (40%, according to the 2012-2015 Performance Contract) is still awarded 
to the traditional ‘SIPPO Pavilion’ approach, which bears the most resemblance to 
CBI’s Export Coaching Programmes in the evaluation time frame. Also, bearing in 
mind that CBI has started out on a similar path towards more systemic interventi- 
ons, this gradual development towards more systemic interventions does not stand 
in the way of a meaningful comparison. 


Additional information on cost drivers has been gathered and analysed, in order to analyse 

and explain variations in cost-per-unit between TPOs: 

• scale of operations (TPO annual turnover, number of simultaneous programmes, 
number of beneficiary companies per programme); 

• geographical spread of operations (number of countries per programme/sector, 
weighted average travel distance per programme); 

• target group characteristics (average amount of employees and turnover per beneficiary; 
% of beneficiaries with export experience at programme entiy); and 

• programme characteristics (average programme duration, average number of fair visits 
per programme, average number of personal visits to beneficiaries per programme). 
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Annex 13 Literature overview of impact studies on export promotion 

Authorand Country Type of assistance Method Positive Positive effects Other effects: Identified key success factors 

date of effect on via intensive 

publication export and extensive 
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Evaluation and study reports of the Policy and Operations Evaluation Department ( 10 B) published 2011-2015 


Evaluation and study reports of the Policy and Operations 
Evaluation Department (IOB) published 2011-2015 

Evaluation reports published before 2011 can befound on the IOB website: 
www.govemment.nl/foreign-policy-evaluationsorwww.iob-evaluatie.nl. 


IOB no. 

Year 

Report 

ISBN 

409 

2015 

Evaluation of the Matra Programme in the Eastern 
Partnership countries 2008-2014 

978-90-5328-475-9 

408 

2015 

Aided trade: An evaluation of the Centre for the 

Promotion of Imports from Developing Countries 
(2005-2012) 

978-90-5328-477-3 

407 

2015 

Opening doors and unlocking potential: Key lessons 
from an evaluation of support for Policy Influencing, 
Lobbying and Advocacy (PILA) 

978-90-5328-474-2 

406 

2015 

Beleidsdoorlichtingvan de Nederlandse humanitaire 
hulp 2009-2014 

978-90-5328-473-5 

405 

2015 

Gender sense b sensitivity: Policy evaluation on 
women’s rights and gender equality (2007-2014) 

978-90-5328-471-1 

404 

2015 

Met hernieuwde energie. Beleidsdoorlichting van de 
Nederlandse bijdrage aan hernieuwbare energie en 
ontwikkeling 

978-90-5328-472-8 

403 

2015 

Premises and promises: A study of the premises 
underlying the Dutch policy for women’s rights and 
gender equality 

978-90-5328-469-8 

402 

2015 

Work in Progress- Evaluation of the ORET Programme: 
Investing in Public Infrastructure in Developing Countries 

978-90-5328-470-4 

401 

2015 

Evaluation of the MDG3 Fund: ‘Investing in Equality' 
(2008-2011) 

978-90-5328-468-1 

400 

2015 

The Only Constant is Change: Evaluation of the Dutch 
contribution to transition in the Arab region (2009- 
2013) 

978-90-5328-467-4 

399 

2015 

Gender, peace and security: Evaluation of the Netherlands 
and UN Security Council resolution 1325 

978-90-5328-465-0 

398 

2014 

Navigating a sea of interests: Policy evaluation of Dutch 
foreign human rights policy 2008-2013 

978-90-5328-460-5 

397 

2014 

Riding the wave of sustainable commodity sourcing: 

Review of the Sustainable Trade Initiative IDH 2008-2013 

978-90-5328-464-3 

396 

2014 

Access to Energy in Rwanda. Impact evaluation of 
activities supported by the Dutch Promoting Renewable 
Energy Programme 

978-90-5328-463-6 

395 

2014 

Strategie bij benadering. Nederlandse coalitievorming 
en de multi-bi benadering in het kader van de EU- 
besluitvorming (2008-2012) 

978-90-5328-462-9 
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IOB no. 

Year 

Report 

ISBN 

394 

2014 

Autonomy, partnership and beyond: A counterfactual 
analysis of policy coherence for Ghana 

978-90-5328-459-9 

393 

2014 

Balanceren tussen koopmanschap en diplomatie. 

Evaluatie van de Netherlands Business Support Offices 
2008-2013 

978-90-5328-458-2 

392 

2014 

Good things come to those who make them happen: 
Return on aid for Dutch exports 

978-90-5328-456-8 

391 

2014 

Useful patchwork: Direct Funding of Local NGOs by 
Netherlands Embassies 2006-2012 

978-90-5328-455-1 

390 

2014 

Investeren in wereldburgerschap. Evaluatie van de 
Nationale Commissie voor Internationale Samenwerking 
en Duurzame Ontwikkeling (NCDO) 

978-90-5328-454-4 

389 

2014 

Op zoek naar focus en effectiviteit. Beleidsdoorlichting 
van de Nederlandse inzet voor Private Sector Ontwikke¬ 
ling 2005-2012 

978-90-5328-451-3 

388 

2013 

Impact evaluation of improved cookingstoves in 

Burkina Faso: The impact of two activities supported by 
the Promoting Renewable Energy Programme 

978-90-5328-449-0 

387 

2013 

Between Ambitions and Ambivalence: Mid-term 
Evaluation SNV Programme 2007-2015 

978-90-5328-448-3 

386 

2013 

Evaluation issues in fïnancing for development: 

Analysing effects of Dutch corporate tax policy on 
developing countries. 

978-90-5328-447-6 

385 

2013 

Economie diplomacy in practice: An evaluation of Dutch 
economie diplomacy in Latin America 

978-90-5328-446-9 

384 

2013 

Achieving universal access to sexual and reproductive 
health and rights: Synthesis of multilateral contribution 
to advancing sexual and reproductive health and rights 
(2006-2012) 

978-90-5328-445-2 

383 

2013 

NGOs in action: A study of activities in sexual and 
reproductive health and rights by Dutch NGOs 

978-90-5328-444-5 

382 

2013 

Buscando novas relacöes : Avaliacao da politica externa 
dos Paises Baixos para a América Latina. Informe 
especial sobre o Brasil 

978-90-5328-453-7 

382 

2013 

En busca de nuevas relaciones: Evaluation de la politica 
exterior de los Paisos Bajos en América Latina. Resumen 
del informe principal 

978-90-5328-450-6 

382 

2013 

Op zoek naar nieuwe verhoudingen. Evaluatie van het 
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Evaluation of Dutch support to Capacity Development. 
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This report presents the findings of the evaluation 
of the Centre forthe Promotion of Imports from 
Developing Countries (CBI) and its contribution to 
Dutch Private Sector Development (PSD) policy 
between 2005 and 2012. The evaluation uses a 
mixed-methods approach to investigate CBI’s 
sectoral export coaching programme and the 
business support organisation development. 

The report concludes that CBI’s sectoral export 
coaching programmes were relevant, though often 
insufficiënt to stimulate companies to start exporting 
to Europe, mostly due to unclear or misguided 
targeting of participating companies and target 


countries. Business Support Organisation 
Development interventions, on the other hand, 
were generally relevant, but there is little 
information on their effect on the PSD goals. 

Although CBI operated relatively independently from 
the ministry during most of the evaluation period, 
the ministry has already become more prescriptive 
in what it expects from the organisation. A results- 
oriented management model with ex post transfer 
pricing and a reinforced monitoring b evaluation 
system are necessary to further improve CBI's 
operating efficiency. 
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